Bridging Gaps Through Technology
Blog Home

Business Hub with Open Source software (Part 2)

In the previous blog we made the case for a business hub for your association (if you haven’t read it, please do). In this blog we will go over implementing the hub with free open source software.

To recap the point we are trying to make, is that associations are moving from monolithic IT platforms to multiple best of breed systems – CRM, LMS, CMS, Community etc. That creates a need to integrate these systems so that they can “talk” to each other. Currently, this is being done on an ad-hoc basis where vendor systems are creating direct links to each other. This creates a “tight coupling” and makes it difficult to swap out systems in the future.  It also fragments your organizations data, and business processes across many different platforms.

The Business Hub we envision

  • Provides a centralized hosting platform to host micro-services
  • Centralizes all integrations across your organization
  • Provides a vendor neutral platform for consolidating data,  business rules and processes

The need to centralize all the data across your enterprise is important for the following reasons

Data Ownership: If your association data is spread across multiple IT systems, is it really your data? We envision creating a centralized data lake, where the data most relevant to your organization is stored.

No Vendor Lock-in: The centralized business hub we envision will go a long way in reducing vendor lock-in.  Since we are able to create a centralized data store and integration layer, you are less dependent on any one vendor. Cases of associations being held hostage to vendor systems are surprisingly common.

“In economics, vendor lock-in, also known as proprietary lock-in or customer lock-in, makes a customer dependent on a vendor for products and services, unable to use another vendor without substantial switching costs.“

– Vendor lock-in (Wikipedia)

Common Data Model

Every IT system models data differently. Salesforce has Contacts, netFORUM calls them Individual and so on. But why should you depend on vendors definition of your associations data?  That causes several issues

  • Staff has to learn different terminologies, which is different from how you conduct your business.
  • All system to system Integrations have to do data translations.
  • Makes pan organization reports and BI dashboard difficult to implement.

This also means when you move your CRM/LMS/CMS, you now have to learn a new data model, re-create your reports, retrain your staff and so on.

Fragmented Data

Common Data Model

In the hub, we will create a “Common Data Model” that most accurately represents data as your association sees it.  This will endure across current and future IT systems.

Implementation View

The problem of multiple IT systems may be new to associations, but its been around for a while in large enterprises (think a large Bank with branches all over the world). Enterprises have used what is know as an Enterprise Service Bus (ESB) to implement this middleware tier.  ESB’s provide dozens of “connectors” and “data translators” that can communicate with a wide variety of platforms and protocols.

Why Open Source?

With open source you get the same feature rich software that you get with commercial solutions, minus some bells and whistles. You also have to depend on the community for support rather than a large vendor.

While there are very powerful commercial ESB solutions like MuleSoft (owned by Salesforce), Boomi (by Dell) etc. These are very expensive and can only be justified if you have a large development team that will be frequently churning out new integrations.

Fortunately there are some very powerful open source ESB’s available. We plan to use Jboss Fuse which is available for free from Red Hat (owned by IBM). It is very popular and has a large pool of developers and companies that work in this technology space. More importantly it has the option of a paid support plan from RedHat. This gives you the option of using Red Hat to support this technology stack in the future, and are not tied to any one vendor.

We also want to keep costs down, since the first couple of implementations we do will serve as a proof of concept.

Data Lake

We plan to use a centralized data store  to store all business relevant data. Amazon Redshift is a cloud based data warehouse solution by AWS. It can store petabytes of structured and unstructured data.

Business Intelligence

There is a whole new generation of excellent cloud based business intelligence tools. This is one area where commercial products are ahead of what’s available as open source, mostly when it comes to ease of use and ramp up time. We particularly like Association Analytics and Looker.

  • Looker in particular provides a powerful data modeling language (LookML). This allows you to model your data in the Common Data Model we discussed above.
  • Association Analytics is built on Microsoft PowerBI and is a product offering by a company that understands associations and can help implement and support relevant business intelligence dashboards.

Apache Superset

There are also very good open source solutions available, and since we don’t mind doing the initial setup, that is what we intend to use for the hub. Our pick is an excellent web based business intelligence application called Apache Superset. You can take a peek at their visualization gallery here.

Business Hub Implementation View

As we begin to implement this stack for some of our clients, we will add further posts on the topic. We will also contribute our Common Data Model to ASAE. If there is a consensus on what a universal association data model will look like, it will go a long way in standardizing data across vendor systems.

The following two tabs change content below.
Manav Kher

Manav Kher

The official gear-head of fusionSpan. Manav has over 14 years of enterprise software development experience. Previously he spent 7 years in various lead development and architect roles at the National Cancer Institute (NCI). Notably, he contributed to the architecture and development of some of the core components of the caBIG infrastructure, for which he received the NCI caBIG Outstanding Achievement Award.
Comment Image

Leave a reply