submit-social-housing-lettings-and-sales-data

2212 Commits

218 Branches

260 Tags

83 MiB

Tree: 8cdab0de29

Commit Graph

Author	SHA1	Message	Date
Phil Lee	45128b7250	Refactor with split of log to CSV by log type (#1593 ) # Context - Partially related to https://digital.dclg.gov.uk/jira/browse/CLDC-2316 - Comprehending sales or lettings bulk upload CSV is enough, comprehending both simultaneously is rather challenging # Changes - Split out test helper class by log type ie lettings/sales	2 years ago
SamSeed-Softwire	321ba71e0c	CLDC-2197 Improve column counting in bulk upload (#1557 ) * feat: add validation for number of field labels * feat: check field labels are numbers * feat: remove validations for number of columns * refactor: rename non_blank_fields_count to valid_field_numbers_count * test: add functions to generate custom field labels/values * test: that extra invalid field labels don't cause issues * test: that removing a valid field label reduces count by 1 * test: that adding a valid field label increases count by 1 * refactor: rename validate_fields_count and wrong_field_count * feat: add validation that max col count not exceeded when no headers * fix: convert numbers to strings in default_field_numbers * feat: add leniency to max cols count (in case of 1 extra col) * test: explicitly set year of bulk upload to be 2022 * test: add/update tests in validator_spec * chore: lint * test: remove tests from csv_parser_spec that were moved to validator_spec * feat: update 2022 csv_parser to work with new validations * refactor: define number of valid 2022 fields in one place * refactor: remove redundant headers definition * feat: update 2022 csv parser to have col flexibility like 2023 * test: for validator 2022 as well as 2023 * test: simplify 2022/2023 logic and improve layout * chore: lint * test: ensure context descriptions start with "when" * refactor: check fields/columns count within csv parser, not validator * test: update 2022 csv parser tests to work like 2023 * chore: lint	2 years ago
Phil Lee	35de06e324	CLDC-2067 Bulk upload for lettings 23/24 (#1417 ) * refactor questions from validator to row parser * able to switch between bulk upload parsers - depending on what year we are processing * spec tooling to support bulk upload multi year * row parser now has year in namespacing * add static data for 2023 row parser * add placeholder to log to csv for specs * bulk upload aribtrary 23/34 column ordering works * bulk upload supports 23/24 without headers * bulk upload 23/24 supports BOM + invalid chars * dupe tests * port 23/24 attributes_for_log * port 23/24 bulk upload validations * force crossover period * tweak max permitted columns * able to return column for given field * work out column for field for errors * add field_4 as 23/24 setup field * remove duplicate method * map schemes and locations correctly * handle arbitrary number of header rows * add missing fields to bulk upload support	2 years ago

Author

SHA1

Message

Date

Phil Lee

45128b7250

Refactor with split of log to CSV by log type (#1593 )

# Context

- Partially related to https://digital.dclg.gov.uk/jira/browse/CLDC-2316
- Comprehending sales or lettings bulk upload CSV is enough, comprehending both simultaneously is rather challenging

# Changes

- Split out test helper class by log type ie lettings/sales

2 years ago

SamSeed-Softwire

321ba71e0c

CLDC-2197 Improve column counting in bulk upload (#1557 )

* feat: add validation for number of field labels

* feat: check field labels are numbers

* feat: remove validations for number of columns

* refactor: rename non_blank_fields_count to valid_field_numbers_count

* test: add functions to generate custom field labels/values

* test: that extra invalid field labels don't cause issues

* test: that removing a valid field label reduces count by 1

* test: that adding a valid field label increases count by 1

* refactor: rename validate_fields_count and wrong_field_count

* feat: add  validation that max col count not exceeded when no headers

* fix: convert numbers to strings in default_field_numbers

* feat: add leniency to max cols count (in case of 1 extra col)

* test: explicitly set year of bulk upload to be 2022

* test: add/update tests in validator_spec

* chore: lint

* test: remove tests from csv_parser_spec that were moved to validator_spec

* feat: update 2022 csv_parser to work with new validations

* refactor: define number of valid 2022 fields in one place

* refactor: remove redundant headers definition

* feat: update 2022 csv parser to have col flexibility like 2023

* test: for validator 2022 as well as 2023

* test: simplify 2022/2023 logic and improve layout

* chore: lint

* test: ensure context descriptions start with "when"

* refactor: check fields/columns count within csv parser, not validator

* test: update 2022 csv parser tests to work like 2023

* chore: lint

2 years ago

Phil Lee

35de06e324

CLDC-2067 Bulk upload for lettings 23/24 (#1417 )

* refactor questions from validator to row parser

* able to switch between bulk upload parsers

- depending on what year we are processing

* spec tooling to support bulk upload multi year

* row parser now has year in namespacing

* add static data for 2023 row parser

* add placeholder to log to csv for specs

* bulk upload aribtrary 23/34 column ordering works

* bulk upload supports 23/24 without headers

* bulk upload 23/24 supports BOM + invalid chars

* dupe tests

* port 23/24 attributes_for_log

* port 23/24 bulk upload validations

* force crossover period

* tweak max permitted columns

* able to return column for given field

* work out column for field for errors

* add field_4 as 23/24 setup field

* remove duplicate method

* map schemes and locations correctly

* handle arbitrary number of header rows

* add missing fields to bulk upload support

2 years ago

3 Commits (8cdab0de2919e1389deab62458866c21c06ca038)