BizTalk BAM - Data Insertion Mechanism

I have explored the way the BAM works in BizTalk , I would like to share the thoughts. Its very interesting as well.
Service Involved
BTSNTSVC.EXE
BizTalk Server Engine
BAM Event Bus Service

Database Involved

BizTalk Management
BizTalk Message Box
BAM Primary import

Storing Data through BizTalk

Role of BizTalk engine
The interceptors get invoked by the BizTalk engine, during execution of Orchestrations or messaging solution
Based on the Tracking profile the interceptor will determine what data to be captured.
At every persistence point the interceptor also send the data to the BizTalk Message box in a binary form, which will be stored in tracking table.

Role of Event Bus Service
The data is retrieved from the Tracking data table (BizTalk message box)
The binary form (which is a serialized .Net Objects), which are rehydrated to objects, Inturn it is converted to sql statements and store the data in the respective activity table in the BAM Primary Import database


Custom Data Insertion

The custom application must use Microsoft.BizTalk.BAM.EventObservation assembly to access BAM Methods of inserting data into a BAM Activity

There are two primary class for inserting data

• DirectEventStream
• BufferedEventStream
  
  DirectEventStream
  The data will be directly written to the BAM Activity Table in the BAM Primary Import database.
  
  BufferedEventStream
  This class work same as same as BizTalk. The class writes the binary blobs into an indirect database (BizTalk Messagebox). Then BAM Event bus will take care of writing the data to the activity table in BAM Primary Import database.
  
  
  What Happen when BAM Primary Import database Goes Offline
  
  I have conducted a simple test on BAM to know what happens
  
  I have created a simple BAM implementation on the sample BizTalk solution where it tracks the messageid and process date.
  
  BAM Definition and Tracking profile has been deployed in the respective database using BM.Exe and bttdeploy.exe tools
  
  Expected Result
  Number of message should be same as the number of record in the BAM Activity Table.
  
  I brought down the BAM Primary Import database to offline.
  Start the BizTalk services and sent 5 message through the BizTalk
  Once the BizTalk process is completed. I made the BAM database online
  I am not able find the data of the last 5 message for the first few min.
  After few min amazingly I am able to view all the 5 message data in the activity table
  
  
  Findings
  01. When BAM Primary Import database is offline the BAM Event Bus Service throws error as
  
  Event Type: Error
  Event Source: BAM EventBus Service
  Event Category: None
  Event ID: 25
  Date: 26/03/2008
  Time: 12:47:18
  User: N/A
  Computer: VLONP034T
  Description:
  Either another TDDS is processing the same data or there is an orphaned session in SQL server holding TDDS lock.A connection was successfully established with the server, but then an error occurred during the login process. (provider: Shared Memory Provider, error: 0 - No process is on the other end of the pipe.) SQLServer: VLONP034T, Database: BAMPrimaryImport.
  
  For more information, see Help and Support Center at http://go.microsoft.com/fwlink/events.asp.
  02. The BAM Event Bus waits for the BAM Primary Import database to be online to insert the data.
  
  
  Based on the above test and Understanding I tried to answer some question
  
  What is the story with writing information/events to BAM when the BAM database is not available (offline, not visible, SQL upgrades etc)?
  
  The BAM Event Bus service will wait for the BAM Database to come online. Then it will send the data from messagebox to the respective activity table
  
  Especially with event streams other than the orchestration one, does the write simply
  fail?
  Its basically based on what class we use to send the data to BAM
  DirectEventStream – if we try to insert the data using the class the. Insertion will simply fail incase of non availability
  
  BufferedEventStream – This class has the ability to write only to the intermediate database. Inserting the data to the BAM database is the job of BAM Event Bus.
  
  
  Even a buffered stream needs to be flushed at some point. Is this true for BAM use both within and without BizTalk?
  
  If we use BAM using BizTalk message box as the intermediate storage, then BAM Event Bus service will take care of the situation.
  If we used direct event stream the insertion will fail simply.