Analyzing data consistency
Nosql (non-relational) database systems have a huge advantage and disadvantage in their loose data structure. On one hand, this allows for much faster development and agility as your project grows. On the other hand, this also increases the chances of data inconsistency.
Of course, we are all great developers using all measures to prevent data inconsistencies. Nonetheless, as if by magic, there are always outliers in our data and we just can’t handle them properly 🙂.
Well, Mingo’s “Schema analyzer” helps with debugging such cases.
Getting to know your issues
First step is to find out whether you do have any issues with your data. Using Mingo’s “Schema analyzer”, you’ll get a quick glance at the state of your data.
You may restrict the analysis using a query selector and limit the number of documents analyzed. Setting the limit to a negative number will analyze the last X documents matching the query.
Dealing with the known data inconsistencies
Once you see the results, you have several ways to act.
First, you can view documents with specific type of data. For example, you can view all documents where a field is a boolean with TRUE value. Or where a field is number.
You may also pull sample values for each value type. For example, show string value samples of a field.
A great feature is “Group by and count” on a field to show value distribution.
Renaming and deleting fields in all documents
If you find that a field is old, useless or an accident, you can delete it from each document by one click. Sometimes, you find a typo in field name. In such case, renaming all such fields in each document can be done by a few clicks in Mingo.
PS: We recommend performing analysis on your local server / computer. Analyzing requires downloading all the data to Mingo and so it depends on your connection speed.