com.alicloud.openservices.tablestore.tunnel.worker

类 TunnelWorker

java.lang.Object

com.alicloud.openservices.tablestore.tunnel.worker.TunnelWorker

所有已实现的接口:

ITunnelWorker

public class TunnelWorker
extends Object
implements ITunnelWorker

TunnelWorker is a full-incremental integrated service built on top of the TableStore data interface, allowing users to easily implement the consumption and processing of both historical and newly added data in a table. The design philosophy of TunnelWorker is to detect active Channels through periodic heartbeat probes (Heartbeat), and to update the status of Channels and ChannelConnects, as well as initialize, run, and conclude data processing tasks. The automated data processing workflow implemented by TunnelWorker is as follows: 1. Initialization of TunnelWorker resources 1.1 Set the state of TunnelWorker atomically from Ready to Started (CAS operation). 1.2 Perform ConnectTunnel operations based on configurations such as HeartbeatTimeout and ClientTag (client identifier) in TunnelWorkerConfig to connect with the Tunnel server and obtain the ClientId corresponding to the current TunnelWorker. 1.3 Initialize ChannelDialer (used for creating new ChannelConnects). Each ChannelConnect corresponds one-to-one with a Channel, and records the checkpoint of data consumption. 1.4 Wrap a data processor with an automatic Checkpoint recording function according to the user-provided data processing Callback and the CheckpointInterval (interval for recording data checkpoints to the server) in TunnelWorkerConfig. For more details, see: ChannelProcessFactory. 1.5 Initialize TunnelStateMachine (which will handle the automation of Channel state machines).

2. Perform Heartbeat at fixed intervals, determined by the heartbeatIntervalInSec parameter in TunnelWorkerConfig. 2.1 Perform a heartbeat request to get the latest available Channel list from the Tunnel server. The Channels will include ChannelId, Channel version, and Channel status information. 2.2 Merge the Channel list obtained from the server with the local memory Channel list, then create and update ChannelConnects according to the following rules: 1) Merge: For Channels with the same ChannelId, the one with the higher version number is considered the latest state and will be directly overwritten. If a Channel does not appear in the local list, it will be directly inserted. 2) Create new ChannelConnect: If there is no existing ChannelConnect corresponding to this Channel, a new WAIT state ChannelConnect will be created. If the Channel's state is OPEN, the data processing cyclic pipeline task (ReadRecords && ProcessRecords) on the ChannelConnect will also be started. For more details, refer to ProcessDataPipeline. 3) Update existing ChannelConnect: After merging, if a ChannelConnect exists for the corresponding Channel, its state will be updated according to the state of the Channel with the same ChannelId. For example, if the Channel is in the Close state, the ChannelConnect state will also be set to Closed, thereby terminating the pipeline task. For more details, see the ChannelConnect.notifyStatus method.

3. Automated load balancing and good horizontal scalability When multiple TunnelWorkers consume the same Tunnel (with the same TunnelId), during the execution of Heartbeat, the Tunnel server will automatically redistribute Channel resources so that active Channels are evenly distributed across each TunnelWorker, achieving resource load balancing. In terms of horizontal scalability, users can easily increase the number of TunnelWorkers, which can be deployed on the same machine or different machines.

4. Automated resource cleanup and fault-tolerant handling 4.1 Resource cleanup: When the client (TunnelWorker) is not properly shut down (e.g., abnormal exit or manual termination), we will automatically help users reclaim resources, including releasing thread pools, automatically calling the shutdown methods registered by users on the Channel, and closing Tunnel connections. 4.2 Fault-tolerant handling: When the client encounters non-parameter errors such as Heartbeat timeouts, we will automatically Renew the connection for the user to ensure that data consumption can continue stably and continuously.

构造器概要

构造器

构造器和说明
TunnelWorker(String tunnelId, TunnelClientInterface client, TunnelWorkerConfig workerConfig)
TunnelWorker(String tunnelId, TunnelClientInterface client, TunnelWorkerConfig workerConfig, IChannelProcessorFactory factory)

方法概要

限定符和类型	方法和说明
void	connectAndWorking()
void	shutdown()

从类继承的方法 java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

构造器详细资料

TunnelWorker

public TunnelWorker(String tunnelId,
                    TunnelClientInterface client,
                    TunnelWorkerConfig workerConfig)

TunnelWorker

public TunnelWorker(String tunnelId,
                    TunnelClientInterface client,
                    TunnelWorkerConfig workerConfig,
                    IChannelProcessorFactory factory)

方法详细资料

connectAndWorking

public void connectAndWorking()
                       throws Exception

指定者:

connectAndWorking 在接口中 ITunnelWorker

抛出:

Exception

shutdown

public void shutdown()

指定者:

shutdown 在接口中 ITunnelWorker