GIS Tools - Data Profiling Report

What Is a Data Profiling Report?

A data profiling report is a comprehensive summary of every column in a dataset, generated automatically without the analyst having to specify which columns to look at. Where the Column Statistics & Histograms tool focuses on one column at a time, a profiling report shows them all at once: a wide grid of mini histograms, value distributions, type detections, null percentages, unique counts, and quality flags. It is the GIS equivalent of pandas-profiling, ydata-profiling, or Great Expectations' built-in profilers.

The purpose of a profiling report is to answer the question "is this dataset usable?" before you commit time to a serious analysis. A new file from an open data portal might have a thousand polygons and forty fields, of which ten are mostly null, five are duplicated, three contain HTML escapes from a bad CSV import, and two are misnamed. A profiling report surfaces all of these issues in one glance, so you can decide whether to clean the data, drop fields, or look for a better source.

The gis.tools data profiling report runs in the browser against the in-memory FeatureCollection. It auto-detects field types (string, number, date, boolean, geometry), computes per-column stats and histograms, flags suspicious patterns (high null rate, single dominant value, mostly empty strings), and renders the result as a scrollable HTML report. You can export the report as a self-contained HTML file for sharing with colleagues.

Profiling is the foundational step of any responsible data workflow. Skipping it means accepting whatever bugs the upstream data has and discovering them only after you have built a model or visualization on top.

How a Profiling Report Is Generated

Field type inference

For each field, the engine inspects the first N (typically 1000) non-null values and applies a type detection cascade: try to parse as boolean, then date, then integer, then float, then fall back to string. The detected type drives which downstream stats to compute.

Per-column statistics

Each numeric column gets the standard descriptive stats (count, mean, median, min, max, stdev, percentiles). Each string column gets unique count, top values, mean length, and presence of suspicious characters (HTML, control codes, leading whitespace). Each date column gets earliest, latest, and date range.

Null and missing analysis

For every column, the engine reports the null count, the empty string count, and the percentage of missing values. Columns with high null rates get visual warnings.

Duplicate detection

The profiler checks for duplicate rows (entire feature equality) and reports the count. It also flags columns where a single value dominates ("95% of rows have value X").

Quality flags

The profiler emits warnings for:

High null rate (>50%)
Single dominant value
Suspected HTML in text fields
Mixed types in a single column
Implausible date ranges
Geometry validity issues

Key Parameters and Options

Sample size

For very large datasets, the profiler can sample N rows rather than scanning the full dataset.

Output format

Reports can render as interactive HTML, printable PDF, or exported as JSON for downstream tooling.

Field include/exclude

Pick specific columns to include or exclude from the profile.

Practical Applications

Open data evaluation

A researcher downloads a city's parcel dataset and runs a profiling report before committing it to a model. They discover that the assessed_value field has 18% nulls (concentrated in tax-exempt parcels), the last_sale_date field has dates ranging from 1700 to 2099 (clearly bad data), and the owner_name field contains HTML entities from a bad CSV export. They decide to clean the data before modeling.

Onboarding inherited datasets

A new GIS analyst joining a team is handed a folder of fifteen project shapefiles. Running profiling reports on each one in five minutes gives them a complete picture of what each file contains, what its quality issues are, and which files are ready to use versus which need cleanup.

Vendor data acceptance

A contract requires the vendor to deliver parcel data meeting specific quality criteria (no nulls in parcel_id, all dates in 2000-2025 range, no duplicates). The contracting agency runs profiling reports on each delivery to verify compliance before accepting payment.

Schema documentation

A data steward generates profiling reports as part of the metadata package for a public release of a dataset, documenting the actual field distributions and quality.

Pre-ML data prep

A machine learning engineer running spatial models needs to identify columns with high null rates, single-value dominance, or other issues that would degrade model quality. A profiling report makes this identification automatic.

Data quality dashboards

A municipal CIO uses periodic profiling reports as part of an internal data quality dashboard tracking the health of authoritative datasets over time.

Reverse engineering legacy data

Working with an undocumented historical dataset, an analyst uses the profiling report to figure out what each cryptic field name actually contains.

Step-by-Step Workflow

Load your dataset into the GeoJSON, KML, Shapefile & GIS File Viewer.
Open the data profiling tool for the active layer.
Wait for the report to generate (takes a few seconds for typical files).
Scroll through the columns and inspect type, null rate, and distribution.
Click on a column for a detailed drill-down.
Read the quality flags at the top for warnings.
Export the report as HTML or JSON for sharing.
Act on findings by cleaning data, filtering, or dropping fields.

Worked Example

A newsroom data team receives a public records request response: a CSV of 47,000 inmate records from a state corrections department. Before publishing analysis, they run a profiling report. The report flags:

inmate_id has 100% unique values (good — primary key intact)
birth_date has dates ranging from 1900 to 2050 (50 records with implausible dates)
release_date has 38% nulls (expected — many inmates still serving)
race has 12 distinct values where there should be 5 (free-text entry inconsistencies)
facility has one value dominating 89% of rows (suspicious — possibly all returned from one facility instead of statewide)

The newsroom contacts the agency for clarification on the facility issue, cleans the date errors, and standardizes the race values before publishing. Without the profiling report, the dominant-facility issue could have been missed entirely, leading to a misleading story.

Common Pitfalls and Gotchas

Sampling bias. If the profiler samples only the first N rows, it may miss issues concentrated later in the file.
Type misdetection. Numeric IDs that look like integers may actually be opaque codes; the profiler treats them as numbers and computes meaningless means.
Date parsing failures. Dates in unusual formats may all be classified as strings, hiding date-related issues.
Encoding issues. Files in Windows-1252 with French accents may show garbled values that look like quality issues but are encoding bugs.
Geometry profile gaps. Most profilers focus on attributes and skip geometry quality checks.
Performance on huge files. Profiling 10 million rows in browser is slow; sample or pre-aggregate first.
Unicode-only fields. A field with only Unicode characters may not surface in profilers tuned for ASCII.

Tips for Best Results

Always profile new datasets before committing to a workflow.
Save profiling reports as part of your metadata documentation.
Re-profile after each cleanup step to confirm issues are resolved.
Use profiling on a sample first, then on the full dataset to verify the sample was representative.
Combine with the GeoJSON Validator & Fixer for geometry checks.
Treat dominant-value warnings seriously — they often indicate filtering bugs upstream.
Compare profiles of the same dataset over time to detect drift.

Comparison with pandas-profiling and Great Expectations

pandas-profiling (now ydata-profiling) is the gold standard for tabular data profiling in Python. It produces interactive HTML reports with everything you would want and more. Great Expectations is a more rigorous data quality framework with declarative test suites.

The gis.tools profiler is closer in spirit to pandas-profiling but adapted for spatial data and the browser environment. It does not have the depth of feature of those Python tools, but it requires no install, no environment setup, and works on shapefiles, GeoJSON, and KML directly.

Performance Considerations

Profiling time scales linearly with rows × columns. For 100,000 features and 30 columns, expect 1-2 seconds. For 1 million features, 10-20 seconds. Memory usage during profiling can spike if you compute histograms for every column at once.

Data Privacy and Browser-Based Processing

The profiling report is generated entirely in your browser. Sensitive attribute values, distributions, and quality flags are never transmitted to a server. This is essential when profiling datasets that contain personal information, business-confidential data, or pre-publication research data.

Related GIS Concepts

Data quality dimensions. Completeness, accuracy, consistency, validity, timeliness, uniqueness — the standard framework for data quality assessment.

Schema inference. Automatically detecting field types from sample values.

Null analysis. Quantifying and locating missing values in a dataset.

Outlier detection. Identifying values that fall outside the expected distribution.

Frequently Asked Questions

How big a dataset can I profile?

Up to a few hundred thousand features comfortably. Beyond that, sample first.

Does it check geometry quality?

Basic checks (validity, empty geometries). For deep geometry QA, use the GeoJSON Validator & Fixer.

Can I export the report?

Yes — typically as HTML or JSON.

How are types detected?

By parsing sample values through a cascade of type tests (boolean, date, integer, float, string).

Does profiling modify my data?

No. It is a read-only operation.

Upload a GeoJSON file