Taking off with Salesforce

Our Salesforce synchronization solution enables automatic sharing of relevant business data across departments and tools. In this article, we describe how the sharing of contract data between the custom web application of the service department and Salesforce allowed for smooth cross-department access and automated usage-based billing.

The Challenge

Salesforce is a Customer Relationship Management system (CRM) running in a cloud. Since first appearing in 1999 it constantly rose in popularity for two main reasons. First, it goes along the general trend towards cloud-based solutions instead of on premise ones. Second, clever extension of the ecosystem surrounding it (for example by buying Heroku, Tableau and Slack just to name a few), made it one of the most flexible SaaS sales platforms.

Here, I want to share how we have built a custom-tailored two-way Salesforce synchronization service, which enabled our client to have a seamless integration between the sales and service team. The service team of our client manages the digital product (ads) on each platform via a custom service application (service app) based on the powerful django framework. The service app controls the content of their ads as well as big parts of the business logic on the platform. On the other side of the equation we have the sales team that closes contracts with customers.

Let me give a concrete example based on a hypothetical company that sells ads in different sizes (small, medium, big, gigantic). Sales won a contract with a customer over 5 units of gigantic and 3 units of small ads for a given price per unit. One small ad has been created and booked on the contract as reflected in the consumed column of the corresponding product type.

The first three columns are provided by the sales team and managed in Salesforce. The consumed column has to be manually updated when ads have actually been created in the service app.

The need of the sales and service team was to have an automatic, reliable way to link the relevant data from the two services. Therefore, each created ad had to be linked to a contract and reported back to Salesforce. This allows the service team to immediately access the information which products are available to a customer as well as the sales team to check how many product units on a contract are used up in their native work environment without any context switching.

Our Solution

Our two-way Salesforce synchronization solution takes care to report the usage of one unit of the corresponding product back to Salesforce. Technically this is achieved by reading and writing the relevant data from Salesforce using its REST API. Every time an ad is created in the service app, it gets enqueued for reporting. A background process permanently processes this queue and immediately reports usages back to Salesforce. It uses the REST API to create an usage entry for each product used by an ad. Such usage entries are owned by the service app and aggregated by Salesforce in order to calculate consumed and available units for each product type on the contract.

There is a two-way synchronization between Salesforce and the service app in place. Contract data is accessible in the service app. Ads created in the service app get reported back to Salesforce.

Our custom solution defines a clear interface to exchange information using Salesforce REST API. Its design is guided by two core principles:

1. Decoupling
2. Data Ownership

While some reasons for decoupling have been described in the previous section, a clear definition of decoupling is still lacking. I consider systems as decoupled when any of them goes down or the connection between them fails, they continue working independently. Only the exchange of data will be delayed until the connection is reestablished.

Data Ownership, the second key principle, means for each data entry there is only one system responsible, that is allowed to change it, while the other system can only read the information. In our case, contract information is read by the service app directly from Salesforce and displayed to the user if available. Since no information is cached, data stays always up to date! Contracts are owned by Salesforce and never changed by the service app.

Even though the product usage table lives in the Salesforce database, it is owned by the service app. This allows service app users to correct mistakes and change selected products or contracts. However, when a contract is fulfilled, its usage entries get locked and any attempt to change them will result in an error. At this point in time, which can be adjusted to the concrete business requirements, Salesforce can rely on the usage data to be immutable. Product usage can be reliably used to create invoices and accounting reports.

Let us break this down to a straight forward example. Assuming our example contract belongs to “Magic Ads Inc” which bills their customer at the end of the month. Now Petra realizes at the mid of the month, that Klaus has wrongly booked 3 small ads instead of 1 gigantic one. She can just like simply correct this mistake in the service app, because of the clear data ownership and the fact that by billing policy the records are not locked yet.

I want to take a second for a technical excursion to describe the key role of using redis for our queue in decoupling and failure recovery. In this example Ad 3 gets saved in the service app and added to the queue. At the same time, the background process picks up Ad 1 and tries to submit the corresponding usage information to Salesforce.

In case of a submission failure of Ad 1, the background process adds it back to the queue:

This mechanism ensures that temporary submission failures do not lead to any data loss. Once the source of the transmission failure of Ad 1 is fixed, the background process will retransmit it. Note, as we add failed ads to the end of the queue a single failing Ad does not delay others.

Paths we did not take

As a developers-first agency, at djangsters we follow a slim approach to infrastructure and move as much as possible of it to a cloud. Given that our provider of choice, Heroku, has been a subsidiary of Salesforce since 2010, we investigate out-of-the-box solutions first. The service app runs on Heroku. It is a slightly customized Django web application, which stores all Ad data in the attached Postgres database.

Heroku Connect for Salesforce allows database tables to be asynchronously synced with Salesforce. The mappings provide direct access to internal Salesforce entries from the Postgres database and vice versa. While this sounds like a great solution by the provider we loved and relied upon for our infrastructure, it has the negative effects of coupling both services on an infrastructure level.

The lack of a clearly defined interface would make Salesforce a hard dependency for the customer service, i.e. the customer service would rely on Salesforce to function. A dependency that massively increases technical complexity as well as limits business choices for our clients. Given that the interaction between Salesforce and the service app has a clear and limited purpose, a clear separation of concerns that a custom solution provides looks like the better approach both from a technical as well as a business point of view.

Conclusion

By having a custom solution with a clear minimal interface we achieve separation of concerns. Service and Sales teams can access relevant information from their usual work environment without time-intensive context switching, but the two systems continue to work independently, e.g. a failure in one system allows the other team to continue working.

Data ownership provides clear responsibilities. This allows for a reliable link between sold products on a contract and consumed product units (ads) on the platform. While errors can be fixed easily until a given point in time, reports and bills are immutable afterwards.

The solution is cost-effective (only one additional background service is necessary) and low maintenance, due to its fully automated process and reliable error recovery. The clear interface allows it to be easily adjustable to changing business requirements on both sides — sales and customer service.