Skip to content

Conversation

@perryr16
Copy link
Contributor

@perryr16 perryr16 commented Oct 28, 2025

Any background context you want to provide?

If a user imports data that does not match the expected data type, SEED will attempt to parse it, and if it fails it will default to None (or blank).

This isn't an issue for Text or extra data fields. Extra data is saved in the db as a string, and anything can be converted to a string.

This behavior isn't obvious to the user during the "Save Mapping" step.

What's this PR do?

  • Checks the incoming file data for nulls after it has been parsed. If blanks are found, an alert will be displayed to the user highlighting which columns contain blanks.
  • The backend function is very quick. Checking a file with 10,000 properties and 50+ canonical columns completes in under 1 second.
  • Backend request is triggered after mapping and before data quality checks
  • Adds a spinner to the 'Save Mapping' button that disappears when null checks have completed. The button is still enabled even if the request hasn't finished.
  • Note that if the incoming file contains blank values, those will still be flagged.

How should this be manually tested?

Import a file that has data type mismatches for numeric canonical columns (ex: Gross Floor Area = 'abc'). Ensure a warning appears during mapping. Clicking "Back to Mapping" will not clear the error, but clicking "Map your Data" will.

What are the relevant tickets?

#5080

Screenshots (if appropriate)

Screenshot 2025-10-28 at 2 02 10 PM

@perryr16 perryr16 added the Feature Add this label to new features. This will be reflected in the change log when generated. label Oct 28, 2025
@perryr16 perryr16 changed the title 5080/pre dqc verify data types Check mapped data for nulls Oct 28, 2025
@perryr16 perryr16 changed the title Check mapped data for nulls Check mapped data for blanks Oct 28, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Feature Add this label to new features. This will be reflected in the change log when generated.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants