...
For transactional publishing, it is broken down into two different cases in order to provide better latency and throughput under different usage situation
Message
...
Write Time Close to Transaction Commit Time
This is mainly the case for messages published from a transaction used in the same JVM process. For example, Worker, Flowlet, Service and Lifecycle methods of MR and Spark. All messages published will be buffered on the client side and only write to the messaging system when the transaction is ready to commit.
...
- Create the row key for the Message Table, using the topic, publish timestamp (same as current timestamp) and sequence id
- Write the row with the payload stored under the payload column (the
pcolumn) and the transaction write pointer under the transaction column (thetcolumn)
Message
...
Write Time is Not Close to Transaction Commit Time
This includes messages published from MR tasks, Spark driver and executors, as well as a lot of messages being published from the same JVM process such that it exceeded the client side buffer (should be a very rare case).
...