* Copy 23 BU files to 24
* Update field mapping
* Add duplicate log error to charges
* Check the sum of charges for duplicates
* update stub
* Only add charges to duplciate hash if they exist
* Rebase change
* Update columns
* Add tcharge back to error mapping
* Add prepare your file
* Add csv parser 24/25 cases
* Fix date check
* Rebase fix
* Empty commit
* feat: base lettings behaviour
* feat: formatting improvements
* Empty commit
* Empty commit
* Empty commit
* refactor: lint
* feat: more page styling
* feat: update tests
* refactor: lint
* feat: add requests spec
* refactor: simplify
* feat: add tests
* feat: copy change to reflect that the count in the header is not necessarily as large as the errors shown (it excludes duplicate errors on fields, and not_answered errors)
* feat: copy changes to sales as well
* feat: remove LegacyBulkUpload MVC
* feat: fix typo
* feat: update tests
* feat: mark additional not_answered sales q
* feat: update routing and only display warnign text when errors will be cleared
* feat: update copy and error rows
* refactor: lint
* feat: don't show soft validations in deletion report
* feat: update routing so deletion report is not shown once logs uploaded
* feat: update tests
* feat: make unique count track repeated errors across rows
* refactor: lint
* feat: match email regardless of casing in bulk upload
* feat: allow null emails
* feat: set encoding by bom for file IO before parsing
* Revert "feat: set encoding by bom for file IO before parsing"
This reverts commit 03d2d17e7b.
* Save total logs count for bulk upload
* Update content of fix choice and confirm pages
* update bulk upload banner
* Rebase changes
* Extract fix choice summary into method
* Add comma
* some minor refactoring
remove methods from child class that replicate methods on the parent class
tidy up check for nil
remove gubbins and inline method body given only used once
* update import services for lettings and sales to import creation method
write tests to cover this
* create sales log field import service and associated spec file, with methods and tests for importing the creation method of logs that have already been imported
* update lettings log field import service and related spec to allow importing creation method of logs
* use the methods dynamically created by active record in all relevant places, removing obsolete methods in teh process.
various tests tweaked to suppor this change.
rake task from another ticket folded into this ticket to prevent merge conflicts
* rename method for ruby conventions
* update PR for altered spec
upload id now decided to be a better indicator of bulk upload status, import service amended accordingly
tests updated in line with this
* update field import services in line with import services to use upload id rather than upload method as the source of truth for how a log was created
* slight refactor to reduce nesting and dodge linter complaints
* minor amendment to log creator spec in bulk upload to use enum dynamic methods
* CLDC-2494: WIP
* CLDC-2494: wip
* CLDC-2494: page work in progress
* cleanup
* Add a path for duplicate logs
* Display all duplicate logs
* Move a test
* Display duplicate check answers for logs
* Add buttons to delete duplicates
* Add a route for sales logs duplicates
* Update duplicated page to work for sales logs
* Update styling
* lint
* Add auth
* Rebase updates
* Remove propcode from dedulication checks
* Update fields to work with supported housing
* Trigger duplicate log check on buyer 1 age not known
* compare correct charges
* Update displayed questions
* BU test
* Put redirect to duplicate logs path behind a feature flag
* More BU tests
---------
Co-authored-by: Kat <katrina@kosiak.co.uk>
* form handler to return all questions from lettings forms for all years with ordering interleaved
functionality and tests
* refactor lettings log csv service and all associated tests
remove methods on log models when we can call them directly on associated models
update job to call the service directly with the collection of logs rather
minor modifications to the sales log csv service
update many test files to test the appropriate logic in the appropriate place
* tidying
final amendments to tests
remove commented code
rename variable
* change the position of the rent value check field in the headers
* CLDC-2492 add creation method field to logs (#1738)
* create migrations to add creation method fields to both log types
* add enum definition to logs for creation method
* upadte csv export services to retrieve creation method values direct from the log, remove methods previously used from the log model
* run migrations to update schema
* ensure that logs created via bulk upload have this set correctly when created
* Renme end_date to new_logs_end_date
* Display change buttons in CYA if the collection is still open for editing
* Allow navigating to question if the collection is still open for editing
* Allow logs to be edited if the date is before edit end date
* Update sales validation to allow editing existing logs
* update tests
* Update edit_end_date
* Update some test wording
* Update new logs end date
* tests
* Validate that correct template for the year is used for sales (with headers)
* Check the correct template is being used without headers
* Check correct template for lettings
* Update csv parser on sales
* Remove redundant methods
* Extract form years
* Reverse year check mathod
* Sends a correct email if there are only soft validations
* Add soft errors valid page
* Add soft errors confirm page
* Confirm the soft validations
* Reuse the how to fix template for check soft validations email
* Update email link
* Move soft validation confirmation to processor
* Correctly set the log status, remove redundant confirm_soft_validations
* Redirect successful upload to logs index and display a success banner
* Implement the soft validations only journey for sales logs
* Display the soft validation errors on the soft-errors-valid page
* Fix page alignment
* Fix tests by mapping housingneeds in csv hepler
* Add the sales soft validations to unpend_and_confirm_soft_validations
* Change naming
* Update method names
* refactor
* undo typo
* Only set the existing soft validations for correct types
* Fix path name
* Add missing error mappings for location fields
* Add missing tests and cancel button
* Change some wording
* Typos
# Context
- https://digital.dclg.gov.uk/jira/browse/CLDC-2344
- So the reported bug and these change do not match 100%
- As the reported bug should have already been resolved in an earlier change as the error message that was reported has already been removed from the system
- These changes remove the check from sales plus a few minor tidy ups
# Changes
- Remove dead code around error threshold which is no longer used or needed
- Inverted negative predicate to positive one, `incorrect_field_count` => `correct_field_count`
- Remove `validate_min_columns` from sales bulk upload
* feat: wip blank fields and dependent fields on upload tos ee if valid and can upload with missing info - this is not the exact ac on the ticket yet
* feat: update seed
* feat: wip commit
* feat: add postcodenk error so can clear on validation
* feat: add postcode validation back
* feat: move la vals to shared and move and add tests
* feat: add correct pluralisation to warning message
* feat: add blank compound invalid fields methods
* feat: update validations
* feat: update pluralisation
* refactor: lint
* feat: clear errors associated with blanked values so log status is set correctly on creation
* feat: validate instead
* feat: avoid duplicated errors
* feat: dont auto-refuse income, different to imports
* feat: validate after every blank method
* feat: delete la validator spec
* tests: update
* refactor: erblinting
* refactor: cleanup
* refactor: move pluralizer to helper
* feat: copy update
* feat: rename
* feat: refactor to avoid redundant re-validations and test
* refactor: rubocop
* test: update
* test: update
* test: update
* update sidekiq
* feat: clear errors
* feat: run clearing twice in case first clear creates different errors
* feat: remove moved file
* feat: undo validation file tweaks as shared/specific overlap could do with a more general refactor
* feat: update tests
# Context
- https://digital.dclg.gov.uk/jira/browse/CLDC-2316
- Implement bulk upload sales for new collection year 2023
- This is a first pass implementation and will probably have some bugs in it and we can address over time
# Changes
- Add CSV parser for sales 2023 to handle CSV structure
- Tweak collection window validation so error now contextual to year selected for upload
- Handle arbitrary ordering of CSV columns
- Fix ordering of errors in report and now ordered by cell
- Added `Upload your file again` link styled as button on error report to match lettings experience
- Update tooling to convert logs to 2023 csv rows with support for random column ordering
# Known issues
- There seem to be some issues with how UPRN is handled if the UPRN cannot be validated.
- For the above I think there is dependency on https://github.com/communitiesuk/submit-social-housing-lettings-and-sales-data/pull/1570 which should clear any errored fields so users can continue to create logs and fix within the service
kosiakkatrina
natdeanlewissoftwire
Jack
Arthur Campbell
Aaron Spencer
Phil Lee