As everybody knows, the world has gone crazy with COVID. My first idea, initially, was to try and make the data about COVID available to everyone through charts and APIs. I started working with MongoDB Atlas and MongoDB Charts which are embedded in MongoDB Atlas. In order to produce those charts, I needed data. I was inspired by John Hopkins University and the charts they have been producing since the beginning of the pandemic and, luckily, they provide a GitHub with a bunch of CSV files. Originally, I thought about using the daily reports and doing an import everyday. However, it turned out that data was not usable as it was (different formats for each report, data written in completely different ways) and because it lacked consistency and predictability, I couldn’t build charts, GraphQL APIs, or REST APIs to be made available to the public.
There was another folder in that repository called the “Time Series” folder. It contains five files about confirmed cases, deaths, and recovered in the US and globally. The files named global mean country level, even sometimes broken down to county or province when the countries are too big. All those files look the same and are consistent in the way the data is organised. The problem I had was that, when I imported the file using Mongoimport, which is the default tool to import CSV files, I ended up with a terrible schema that I don’t want in my MongoDB cluster. The date is a key and not a value, which means I can’t put a filter on it. The localisation should be a GeoJson point which I could reuse in my MongoDB Charts and all the MongoDB queries or indexes. There is also a lookup table that provides more information about all the countries, counties, states and regions. You can get the population per area, ISO codes, etc, which led me to want to regroup all that information together. I applied some Python magic and end up with this:
Once I had data that worked, I built collections based on that (global, US only, countries summary, etc). From those collections, I’ve built a REST API and a GraphQL API in MongoDB Realm and some charts in MongoDB Charts, and today I want to show you how to reproduce the GraphQL API on MongoDB Atlas (the MongoDB fully-fledged data platform).
GraphQL Demo
MongoDB Atlas is updated every hour and is where you can build your cluster and safely host your data. There is a place where you can store your charts but what really interests us today is MongoDB Realm, which is a backend serverless platform that lives on top of MongoDB Atlas. We will create a new application by clicking on the green button on the top right-hand side and call it APIdays Paris. Next, we link it to our database using an existing MongoDB Atlas Data Source (choose covid-19) and, in the advanced configuration at the bottom, we’ll make sure it is also deployed in Ireland as my cluster is also in Ireland and having both the back end and the actual cluster next to each other will reduce the latency. Then you create a Realm Application, which takes a few seconds and you are in! Take some time to visit and have a look around to familiarise yourself with the layout. On the left-hand side, you have tools allowing you to personalise the app, such as adding an authentication system, creating a GraphQL API, functions, and triggers. You can also have HTTP endpoints, which is a way to host the REST API and you can also host a full website.
Focusing now on the GraphQL part, I obviously need to start by generating JSON schema for my collection. It shows a list of collections from which I can select the one I want and then decide on a few permissions templates (read and write, read-only, etc). As I want to create a read-only REST API, I select read-only and then configure the collection. Once this is done, you move to the schema tab, click on the generate schema button choosing one of the options available at the bottom. You can bulk generate all the collections but in this case, I choose to only generate the global collection. Once you have the schema, you can tweak it as you want. You just save it and go back to GraphQL to discover you have a new interface. Using the schema you have just created, you have a pre-built query that you can run using the arrow button and that’s it. You have a GraphQL API in production. If you want to deploy it, you need to be authenticated, which you can do by choosing several methods such as email & password, Facebook, google, etc. Once that’s saved, you just need to copy the app ID on the top left-hand side, replace the app ID in the query, use that to create an anonymous user, get an access token and a refresh token. Now you can just add this token in the query.
If you want to add an extra layer of security, you can go back to authentication, choose to request email and password, deploy again, and then add new users. The last thing I wanted to highlight because it is brand new and has only been added two weeks ago, is the data API, directly in Atlas. It is an out-of-the-box default REST API that does read and write. It’s still in preview, and we are still improving it and working on it so don’t hesitate to provide feedback.
- https://cloud.mongodb.com/
- https://github.com/mongodb-developer/open-data-covid-19
- https://developer.mongodb.com/article/johns-hopkins-university-covid-19-data-atlas
- https://developer.mongodb.com/article/johns-hopkins-university-covid-19-rest-api
- https://developer.mongodb.com/article/johns-hopkins-university-covid-19-graphql-api
- https://charts.mongodb.com/charts-open-data-covid-19-zddgb/public/dashboards/60da4f45-f168-434a-82f6-d37ce88ff9ea
- https://community.mongodb.com/
Q&A Section
Q: How can we learn more about MongoDB?
A: We have MongoDB University which provides free courses and allows you to become certified.
