Data infrastructure

The Toucan’s way: a user centric approach to data management for analytics

Like you, we strive to deliver a frictionless user experience. It means we should not start from technology aspects but from user’s requirements.

And the first one is: every feedback should be given in less than 1 second.

So the question you have to answer here is: “Can we guarantee a response time below 1 second from our data infrastructure?”

When it comes to analytics and data visualization, it implies 2 things:

  • data is pre-aggregated
  • data is indexed following a user and business centric approach

If you can, go to Leverage your existing analytics infrastructure with Toucan.

If not, don’t worry, here is how we help you do it: Bootstrap, build and scale an analytics infrastructure the Toucan’s way.

Data-Connection

Data-Connection

Leverage your existing analytics infrastructure with Toucan

Warning

The pre-requisites for this implementation is that you can guarantee a response time below 1 second on your data infrastructure.

Implementation

Connect Toucan directly to your infrastructure through a live data connexion

Data storage

You decide which data can be stored or cached in Toucan Toco. It could be none or a part of it. If you’ve implemented a rich enough authentication layer, Toucan Toco will then only query your system with the current user’s authorizations. Toucan will not be able to access this data on its own, you have full control on authoriztaions over your data.

Data update

Every time a visualization is displayed, the embed will directly query your data infrastructure.

Data preparation

As data is not stored in Toucan, our data preparation tool Visual Query Builder is not yet available. Instead, you have to use Toucan Post Process operations to perform it.

Bootstrap, build and scale an analytics infrastructure the Toucan’s way

If you don’t have a performant analytics database, how can you still offer a frictionless analytics experience to your users?

A performant analytics database has 3 main characteristics:

  • Historization: In production databse, you’ll undoubtedly keep that last data up-to-date (e.g: product 1 at 42$). But in order to follow changes and make comparisons, you’ll need to keep the history (e.g: I want to compare the price of product 1 from April with May)
  • Aggregation: It should also stored data already aggregated
  • Quick response time: In order to have a nice user experience, it should return the data quickly (ie: it should not crash or freeze my application because my requester more than 1 second to complete)

Why is it an issue if I don’t already have a dedicated analytics database?

Let’s take the example of an e-commerce website, called Acme, with an SQL production database and a production API.

Acme can easily change the price of a given product but may not be able to retrieve data’s history on the price. That’s because an SQL database shouldn’t be used to do so.

But imagine, Acme does it anyway. Now, we want to aggregate on the fly all Acme’s product by status (Ongoind Shipping, Delivered, …) for each months. Depending on the number of products and if we have historization on it, it’s pretty clear that SQL database won’t have a quick reponse time.

It will impact the end user experience.

So how can Toucan’s way help face those challenges?

Step 1: adopt a design first approach

Data projects can be long and costly. By adopting a design first approach, you make sure that you focus your investment where it creates the more value for your users.

We’re not talking about protyping here. Prototyping is great BUT at some point you have to throw it all and build it again for production.

Our approach is akin to bullet-traced developement. The first version you’re going to build will then be plugged to your infrastructure at very limited costs. So you keep everything you’ve built.

At this stage the tech team can provide the builders with a simple dump from its production database or a csv extract.

Step 2: use Toucan Data Store to build an analytics database

Now that you know what you need to build, Toucan provides you with the underlying data schemas and MongoDB powerful aggregation framework to implement it.

Those schemas will help you:

  • build datasets from your production database that will be made available to small app designers
  • identify business segments and relevant MongoDB indexes to ensure quick aggregations and store it, in order to give non-technical users, easy data to build stories with.
  • provide you with response time feedback so you know where further optimization are needed.
  • you can schedule your data fetching from your production database to keep history in our Data Store.

Step 3: scale your infrastructure in the cloud!

The MongoDB provided by Toucan performs well for less than 10 GB of data.

When you reach that threshold, migrating to a cloud based providers such as Mongo Atlas is a no-brainer. And then you can scale your infrastructure as you need. No strings attached.