Optimization Cheat Sheet¶
Here are a few things you should have in mind regarding data in Toucan! 💡
How do you make sure that your application is optimized to run in the long term?¶
Data prep is the key¶
Factorized your pipeline¶
If with all the optimizations that you made, the loading time is still a bit high, it’s time to add mongo indexes!
Don’t forget to measure the improvements with the network tab
Checklist to make sure your application is optimized¶
Queries in general:¶
- ✔️ Check if queries are filtered first.
- ✔️ Check if they return only what is displayed on the screen.
- ✔️ Check if part of the query could be done in dataprep and thus increase display speed.
- ✔️ No hard coded values: always prefer smart rules (ex: argmax on year instead of 2021).
Check the data pipeline: is it clear and easy to read ?¶
- ✔️Clear domain names.
- ✔️ Keep only used domains.
Check the date requester construction (if you are using the old one):¶
- ✔️ Is it prepared with dataprep ? (because treatment will otherwise be played at each screen loading).
- ✔️ Enough date format to use in all screen queries filtering.
- ✔️ Anticipate a year + 1: will it keep on working ?
- ✔️ Nice to have: year -1 or month -1 date column if you need it in your screen queries calculations.
Check report requester construction:¶
- ✔️ Is it prepared with YouPrep ? (because treatment will otherwise be played at each screen loading).
- ✔️ Nice to have: a column for the order (tip: use conditional step to create it).
- ✔️ Nice to have: if you need it, a “type” column is useful, several if you have a hierarchical report (children type, parent type).
- ✔️ Nice to have hierarchical: have both one parent/one children column and intermediary levels in columns for each children (tip: can be done with YouPrep as dataprep : rollup + join).
- ✔️ How long do I think this data architecture can last before being improved? (limits in data volume, in queries preparation?).
- ✔️ Data validation is in place.
- ✔️ Check if the screens are fast enough.
- ✔️ Check if the home is fast enough.
- ✔️ Check if mongo indexes are implemented (if needed).
- ✔️ Check if requesters should be used instead of filters.
- ✔️ Are my screens easily readable on mobile?