What is Microsoft BizTalk Server? Introduction in 10min.
Table of Contents
Microsoft BizTalk Server
If you wanted to explain Microsoft BizTalk Server to a technology guy, the answer will be:
Microsoft BizTalk Server is a middleware product from Microsoft that helps to connect various systems together.
Let’s take an example: If you look at any modern organization it will run its operations using a variety of software products. SAP for their ERP needs, Salesforce for their CRM needs, Oracle for their Database needs, + tons of other homegrown systems like HR, Finance, Web, Mobile, etc.
At one point in time, these systems need to talk to each other, for example, customer data that’s residing in your SAP system may be required in your CRM system (Salesforce). In a similar way, the contact details you collected from your company website need to go into a few backend systems like CRM, ERP, Marketing, etc.
This business need can be addressed in a layman’s way by allowing each system to talk to all dependent underlying systems. From our example, the web will have a piece of code that will update contact details in CRM, ERP, Marketing system, etc (the similar way each system will have its own implementation to update relevant systems). If you go down this route you will end up with 2 major issues. One creating a spaghetti of connections/dependencies between various systems, and second, whenever a small change is required you need to touch multiple systems. There are various other challenges like understanding the interfaces of all the underlying systems, transport protocol, data formats, etc.
Products like Microsoft BizTalk Server (there are other vendors like Tibco, Mulesoft, and IBM Websphere Message Broker), etc solve this problem of being a middleman.
When you use Microsoft BizTalk Server, all the systems talk to only one central system i.e. BizTalk server and it’s the responsibility of the BizTalk server to deliver the message to the corresponding underlying system. It takes care of the various challenges I highlighted earlier.
In a real-world example, imagine Microsoft BizTalk Server as a postman delivering letters. It’s impossible for all of us to go and deliver letters to each address, hence we post them it the post office and they take care of delivering them.
If you see BizTalk from a bird eye view you will find it middleware. A middleman who works as a communicator between two businesses, systems or/and applications. You can find many diagrams on the internet that shows it as a middleman or tunnel that use by two willing systems to exchange their data.
What is BizTalk Server?
As per Microsoft Docs’ statement, “Combining different systems into effective business processes is a challenging problem. Accordingly, BizTalk Server includes a range of technologies. It provides the ability to communicate with a range of other software. By relying on adapters for different kinds of communication, the engine can support a variety of protocols and data formats, including Web services and many others.”
Use of Microsoft BizTalk Server
If you want to see it more technically then you can say, it is an integration or/and transformation tool. With the robust and highly managed framework, BizTalk has infrastructure that provided communication channels with capabilities to provide desired data molding and transformation. In an organization, sometimes data exchange with accuracy with minimum effort is the desired goal. Here BizTalk plays a vital role and provides services to exchange data in the form they can understand. It makes applications transparent to each other and allows them to send and receive information in the way they used to send it regardless of what kind of candidate is existing for the information.
If you go further deep you will find it messaging engine based on SOA. To make BizTalk work for every plate form Microsoft used XML. People say BizTalk only understands XML. Not purely, you can also send binary files through BizTalk. But when you want functionality, logging, and your business rules then you can only play in XML. BizTalk is SOA (Services Oriented Architecture), many types of adapters are available to interact with different kinds of systems and can be changed and configured at the administration level.
So, this is creepy stuff that can anyone say about BizTalk to impress his Boss, or to clear his interview as an introduction. Now let’s go into deep and taste BizTalk further.
Components of Microsoft BizTalk Server
Next, I can talk about Message Box. Take a look at following image that you can find on many other websites,
Four major components can be seen
- Receive Port
- Message Box
- Send Port
- Orchestration
Yeah your guess is right, Receive port is where we receive the request, and send port is where we send the request but what is the message box and Orchestration?
First, if we talk about the flow of execution. The message reaches receive port through the adapter we configure, it reaches here as we configure its receive location and adapter. Then it goes through the pipeline toward the message box. From the message box, the message is sent to the port subscribe for this message. It can be more than one port. The message is published in the message box to all recipients. As the port is identified, the message is sent to orchestration for the port and again comes back to the message box and then sends the port’s map and pipeline. Finally, the adapter sends a message about where it should go. Maps are optional, it is according to need. A pipeline is compulsory, but few built-in pipelines are available and you can use them if you do not want to do anything in pipelines.
Message box is simply we can say a SQL Server Database. Here we define the message arrive should be sent to which port. Message arrived with unique signature; we call it message namespace. This namespace should be unique in subscription. It helps the BizTalk to transport message to correct location. There is the other type of subscription of message and also untyped messages that are routed on the basis of data that contain but those are beyond scope of overview.
Receive port is further explode into receive location, pipeline and maps. Receive port execution is done such manner, first adapter then pipeline and then port. Receive location is a here as a separate artifact. Configuration of receive location is important to initiate service. Here we define what adapter will be used to get message. Further we can introduce pipeline here. Pipeline is used to perform any operation prior to send message to message box. Normally we used to disassemble any file.
Then inbound maps are faced, here we can mapping operation. BizTalk Mapper is a tool ship with BizTalk Server with vast variety of mapping operations.
Orchestration is an implementation of your business logic. Microsoft provide BizTalk template that will install in Visual Studio having GUI interface for orchestration, map and other component.
Message sent to orchestration on the basis of subscription and then again to Message Box to noted down the changes done in orchestration and finally to send port. On send port we also have map, pipeline and adapter to perform any changes at sending end. This execution occurs in reverse order as compare to receive port means first outbound maps then pipeline and then adapter.
This is execution of any message through BizTalk.
Why do people still use Microsoft BizTalk Server?
In Microsoft stack, when it comes to Integration BizTalk Server is the key product. These days the existence of isolated applications are diminished, every single application needs to talk to other applications.
If you don’t have something like BizTalk Server, soon these applications integrations are going to create lot of point to point connections creating a spaghetti mess in your organisation.
BizTalk Server is not the only player in the market, there are various other competitors like Tibco, Oracle ESB, IBM WebSphere Message Broker, IBM ESB etc. If you are primarily Microsoft focused company, then BizTalk Server will be your key integration solution.
Microsoft is continuing it’s investment in BizTalk Server, since 2000 it released one major version every 2 years once (2000, 2002, 2004, 2006, 2006 R2, 2009, 2010, 2013, 2013R2, 2016 and 2020). In last few years, the cloud became one of the key topic for discussion. It’s natural for integration products to talk to cloud in the future. BizTalk Server 2020 rightly position itself to make it seamless to talk to cloud, example out of the box it comes with set of Cloud related adapters to talk to Windows Azure Service Bus and logic apps.
Why can Microsoft BizTalk Server be considered the best?
Below are few points that can be considered for considering BizTalk as one of the best Integration tool:
- Declarative Programming Model i.e. in contrast to Imperative Programming which implements algorithms.
- Adapter Based Communications, BizTalk provides multiple adapters to connect to almost all the applications/systems(like SQL,SAP, SharePoint, FTP etc.)
- Strong support for transformation, Maps in BizTalk are used for message transformation where it provides functions zipped into a method box called functoids.
- Flexible and easy deployment of BizTalk applications, we can also achieve silent deployment using power-shell script. BizTalk applications can be deployed with use of BTDF, .bat files/command prompts, MSI packages.
- BizTalk gives the developer flexibility to create custom components which can be pipeline components, functoids, adapters, bindings etc.
- BizTalk provides Business Rule Engine for automation of Business process and Business logic .
- BAM, helps in real time monitoring can be used with BizTalk for tracking data following through multiple applications.
- BizTalk transfer data in XML format, it is also user friendly.
Which are the companies using Microsoft BizTalk Server?
These are the figures :
- 10,000+ BizTalk customers worldwide
- 23 of 27 EU member governments use Microsoft BizTalk to provide more efficient government services
- 81% of the Fortune Global 100 use BizTalk
- 6 of the 8 largest U.S. pharmaceutical companies
- 4 of the 5 largest electronics parts manufacturers
- 9 of the10 largest U.S. telecommunications companies
- 9 of the 10 largest U.S. Aerospace and Defense Companies
- 5 of the 8 largest U.S. chemical companies
- 4 of the 5 largest railroads in the U.S.
- 9 of the 10 largest insurance companies in the world
- 12 of the 15 largest Retailers in the World run Microsoft BizTalk
- 5 of 10 largest Hotel Chains in the World with over 2 Million rooms use Microsoft BizTalk
Future for Microsoft BizTalk Server Developer
Any technology is evolving one, you need to be prepared to adapt changes.
As someone who is working on BizTalk, your primary skill set is solving “integration problems”, BizTalk is just a tool to achieve it. Yes you’ll know in-depth about how BizTalk is working, you’ll be able to build solutions faster using BizTalk, however the core competency you’ll be gaining is how to solve Integration problems, that’s something no one can replace.
As a next step you need to keep your eyes open on how Integration technology is evolving in the Microsoft stack, you need to expand your horizon with evolving technologies like Azure LogicApps, API apps, Service Bus, Event Hubs, IOT suite etc which are all related.
Slowly the world of integration is moving from connecting just bunch of on-premise legacy systems to much wider scope of SaaS/PaaS connectivity, connecting Salesforce cloud to Office365.
How to integrate BizTalk from Kafka? please let us know how to proceed.
Is it possible to connect BizTalk from Kafka? if possible please let me know.
Hi Prasad, Thank you for your comment.
As of now(till today) there is out of box adapter not available in BizTalk which can be useful to integrate with Kafka. Though you can purchase the 3rd party adapters which will be helpful and for that you have to connect with their team.
Also, one of the community users has already raised a request to have this Kafka adapter inbuilt and if you want can participate in the discussion.
https://github.com/microsoft/Integration/issues/22
Alternatively, I would suggest using Azure LogicApps/Function, where you can integrate with Kafka.