Fetching Data

How API requests work

Each time the data should be fetched, the code sends a request to the API endpoint (server) and gets a response. If it is positive, the server sends the data back in a JSON format. This interaction is captured in the image below.

../_images/api_schema.png

Fetching data from multiple endpoints

In the case of this application, there are 10 endpoints (each one corresponds to a table in the database), each serving different data for a single appliance type. These endpoints are:

  • Elektro celkem

  • Klimatizace

  • Osvit

  • Rekuperace

  • Teplota venku

  • Topeni voda

  • Topeni

  • Topny kabel

  • Vitr

  • Zaluzie

In the fetch_data.py module, the fetch_data function can take one of these endpoint names as an argument, and using the authentification object described in the API connection section, send a request to the relevant endpoint to fetch a data snapshot. If the request succeeds, this data is then parsed and saved into memory for further use.

Validation

To ensure that the API returns data in an expected format, a short validation pipeline is in place within the fetch_data function. This checks that the data structure has not changed since the writing of the script, and if it has, throws an error.

Note

This part of the application is rather prone to breaks, as as it relies heavily on the source API not changing its format. Despite there being several backup solutions for database saving and database handling later in the application, any modifications in the API can cause the data fetch script to fail. As such, ensure that any API changes are communicated ahead of time, and accounted for in the script.