Optimization Cheat Sheet

Here are a few things you should have in mind regarding data in Toucan! 💡

How do you make sure that your application is optimized to run in the long term?

Data prep is the key

Make sure that all of your heavy data transformations are done in data preparation.

Factorized your pipeline

Make sure that your data preparation is factorized. Why do the same operation 10 times when you can only do it once right?
If your loading time is still a bit long, go to the network tab of your developer tool, and check where the loading time is high, and how long is the loading time. Therefore, you will be able to improve your query and check later if the loading time improved.

Mongo indexes

If with all the optimizations that you made, the loading time is still a bit high, it’s time to add mongo indexes!

Note

Don’t forget to measure the improvements with the network tab

Checklist to make sure your application is optimized

Queries in general:

  • ✔️ Check if queries are filtered first.
  • ✔️ Check if they return only what is displayed on the screen.
  • ✔️ Check if part of the query could be done in dataprep and thus increase display speed.
  • ✔️ No hard coded values: always prefer smart rules (ex: argmax on year instead of 2021).

Data architecture:

Check the data pipeline: is it clear and easy to read ?

  • ✔️Clear domain names.
  • ✔️ Keep only used domains.

Check the date requester construction (if you are using the old one):

  • ✔️ Is it prepared with dataprep ? (because treatment will otherwise be played at each screen loading).
  • ✔️ Enough date format to use in all screen queries filtering.
  • ✔️ Anticipate a year + 1: will it keep on working ?
  • ✔️ Nice to have: year -1 or month -1 date column if you need it in your screen queries calculations.

Check report requester construction:

  • ✔️ Is it prepared with YouPrep ? (because treatment will otherwise be played at each screen loading).
  • ✔️ Nice to have: a column for the order (tip: use conditional step to create it).
  • ✔️ Nice to have: if you need it, a “type” column is useful, several if you have a hierarchical report (children type, parent type).
  • ✔️ Nice to have hierarchical: have both one parent/one children column and intermediary levels in columns for each children (tip: can be done with YouPrep as dataprep : rollup + join).
  • ✔️ How long do I think this data architecture can last before being improved? (limits in data volume, in queries preparation?).
  • ✔️ Data validation is in place.

Loading speed:

  • ✔️ Check if the screens are fast enough.
  • ✔️ Check if the home is fast enough.
  • ✔️ Check if mongo indexes are implemented (if needed).
  • ✔️ Check if requesters should be used instead of filters.

Mobile use:

  • ✔️ Are my screens easily readable on mobile?