Analyzing data consistency
Nosql (non-relational) database systems have a huge advantage and disadvantage in their loose data structure. On one hand, this allows for much faster development and agility as your project grows. On the other hand, this also increases the chances of data inconsistency.
Of course, we are all great developers using all measures to prevent data inconsistencies. Nonetheless, as if by magic, there are always outliers in our data and we just can’t handle them properly.
Well, Mingo’s “Schema analyzer” helps with debugging such cases.
Getting to know your issues
The first step is to find out whether you have any issues with your data. Using Mingo’s “Schema analyzer”, you’ll get a quick glance at the state of your data.
You may restrict the analysis using a query selector and limit the number of documents analyzed. Setting the limit to a negative number will analyze the last X documents matching the query.
Dealing with the known data inconsistencies
Once you see the results, you have several ways to act.
First, you can view documents with a specific type of data. For example, you can view all documents where a field is a boolean with a TRUE value. Or where a field is a number.
You may also pull sample values for each value type. For example, show string value samples of a field.
A great feature is “Group by and count” on a field to show value distribution.
Renaming and deleting fields in all documents
If you find that a field is old, useless or an accident, you can delete it from each document with one click. Sometimes, you may find a typo in a field name. In such a case, renaming all such fields in each document can be done with a few clicks in Mingo.
PS: We recommend performing analysis on your local server/computer. Analyzing requires downloading all the data to Mingo, therefore processing time will depend on your connection speed.