* Add in past end dates for testing
* Temporarily disable test marked to delete at year end
* Remove one off reinfer_local_authority task
* Update validator_spec
* Set 2024 date in form_handler_spec to be during crossover period as needed
* Use bulk_upload.year_combo for comparison in request tests to avoid year dependancy
* Update BU log creator specs for 2024
* Use year combo function in bulk upload mailer tests
* Refactor lettings validator_spec
* More fixes
* More work on bu validator specs - mostly sales
* Remove pre 2023 test
* More use of bulk_upload.year_combo in request tests
* Fix lint
* Tweak bulk upload error row component tests for year changes
* Further fixes
* Fix 2023 lettings row parser spec
* Sales log to csv fix
* Refactor BU processor tests
* Fix field number row identifier
* Fix linting
* More years in request spec
* fix
* Don't use db unnecessarily in financial validations spec
* Fix sale date changing 2024 -> 2023 test
* Add tests for bulk_upload.year_combo
* Update bu factory year specification
* Refactoring
* Linting
* Don't use helper in factory
* Remove new 2023 specific test
* Remove dummy end dates
* Empty
* CLDC-3345 Rename created_by to assigned_to (#2372)
* Rename created_by to assigned_to
* Replace created_by with assigned_to
* Update created_by to assigned_to in exports, remove blank assigned to
* CLDC-3345 Add and set created_by fields (#2373)
* Add created_by
* Update existing created_by values
* Set created_by on single log
* Set created_by on BU
* Add created_by to exports
* feat: update since last merge
---------
Co-authored-by: natdeanlewissoftwire <nat.dean-lewis@softwire.com>
* Fix papertrail create version
---------
Co-authored-by: natdeanlewissoftwire <nat.dean-lewis@softwire.com>
* feat: only show editable duplicate sets for user and org
* feat: simplify and refactor
* feat: update tests
* feat: add and update tests
* feat: reset not routed questions twice to catch newly not routed questions after the first set are cleared
* Revert "feat: reset not routed questions twice to catch newly not routed questions after the first set are cleared"
This reverts commit 3007284670.
* TEST FOR PO TO BE REVERTED
* TEST FOR PO TO BE REVERTED (set back to dec 31)
* TEST UPDATE TESTS
* REVERT THIS TEST UPDATE
* REVERT THIS TEST UPDATE
* feat: timecop tests
* refactor: lint
* feat: add update methods
* feat: update tests (and TO BE REVERTED: change collection year)
* feat: update tests
* feat: update tests
* feat: update form end_date tests REVERT LATER
* feat: update tests
* refactor: lint
* feat: revert to previous end dates now passed PO review
* feat: use scoping to simplify editable set logic
* some minor refactoring
remove methods from child class that replicate methods on the parent class
tidy up check for nil
remove gubbins and inline method body given only used once
* update import services for lettings and sales to import creation method
write tests to cover this
* create sales log field import service and associated spec file, with methods and tests for importing the creation method of logs that have already been imported
* update lettings log field import service and related spec to allow importing creation method of logs
* use the methods dynamically created by active record in all relevant places, removing obsolete methods in teh process.
various tests tweaked to suppor this change.
rake task from another ticket folded into this ticket to prevent merge conflicts
* rename method for ruby conventions
* update PR for altered spec
upload id now decided to be a better indicator of bulk upload status, import service amended accordingly
tests updated in line with this
* update field import services in line with import services to use upload id rather than upload method as the source of truth for how a log was created
* slight refactor to reduce nesting and dodge linter complaints
* minor amendment to log creator spec in bulk upload to use enum dynamic methods
* form handler to return all questions from lettings forms for all years with ordering interleaved
functionality and tests
* refactor lettings log csv service and all associated tests
remove methods on log models when we can call them directly on associated models
update job to call the service directly with the collection of logs rather
minor modifications to the sales log csv service
update many test files to test the appropriate logic in the appropriate place
* tidying
final amendments to tests
remove commented code
rename variable
* change the position of the rent value check field in the headers
* CLDC-2492 add creation method field to logs (#1738)
* create migrations to add creation method fields to both log types
* add enum definition to logs for creation method
* upadte csv export services to retrieve creation method values direct from the log, remove methods previously used from the log model
* run migrations to update schema
* ensure that logs created via bulk upload have this set correctly when created
* Sends a correct email if there are only soft validations
* Add soft errors valid page
* Add soft errors confirm page
* Confirm the soft validations
* Reuse the how to fix template for check soft validations email
* Update email link
* Move soft validation confirmation to processor
* Correctly set the log status, remove redundant confirm_soft_validations
* Redirect successful upload to logs index and display a success banner
* Implement the soft validations only journey for sales logs
* Display the soft validation errors on the soft-errors-valid page
* Fix page alignment
* Fix tests by mapping housingneeds in csv hepler
* Add the sales soft validations to unpend_and_confirm_soft_validations
* Change naming
* Update method names
* refactor
* undo typo
* Only set the existing soft validations for correct types
* Fix path name
* Add missing error mappings for location fields
* Add missing tests and cancel button
* Change some wording
* Typos
* feat: wip blank fields and dependent fields on upload tos ee if valid and can upload with missing info - this is not the exact ac on the ticket yet
* feat: update seed
* feat: wip commit
* feat: add postcodenk error so can clear on validation
* feat: add postcode validation back
* feat: move la vals to shared and move and add tests
* feat: add correct pluralisation to warning message
* feat: add blank compound invalid fields methods
* feat: update validations
* feat: update pluralisation
* refactor: lint
* feat: clear errors associated with blanked values so log status is set correctly on creation
* feat: validate instead
* feat: avoid duplicated errors
* feat: dont auto-refuse income, different to imports
* feat: validate after every blank method
* feat: delete la validator spec
* tests: update
* refactor: erblinting
* refactor: cleanup
* refactor: move pluralizer to helper
* feat: copy update
* feat: rename
* feat: refactor to avoid redundant re-validations and test
* refactor: rubocop
* test: update
* test: update
* test: update
* update sidekiq
* feat: clear errors
* feat: run clearing twice in case first clear creates different errors
* feat: remove moved file
* feat: undo validation file tweaks as shared/specific overlap could do with a more general refactor
* feat: update tests
# Context
- Partially related to https://digital.dclg.gov.uk/jira/browse/CLDC-2316
- Comprehending sales or lettings bulk upload CSV is enough, comprehending both simultaneously is rather challenging
# Changes
- Split out test helper class by log type ie lettings/sales
* add first page for bulk upload resume journey
* bulk upload resume handles upload again
* add confirm page to bulk upload resume journey
* replace placeholder count with correct value
* apply recommendation for bulk upload resume choice
* add how to fix bulk upload mailer
* integrate new bulk upload approve journey
* add missing bulk upload error mappings
* remove test
* prevent approve being called multiple times
* bulk upload creates invisible logs ahead of time
* work invisible logs into bulk upload flow
* sort errors so deterministic
* remove unused ensure
* remove expected_log_count and processed
- these fields are no longer used or needed
* introduce pending status
* swap visible for pending logs
* only show visible lettings logs
* hard code status filters
* remove unused model methods
* only show visible sales logs
* form controller ignores hidden logs
* locations and schemes only affect visible logs
* refactor questions from validator to row parser
* able to switch between bulk upload parsers
- depending on what year we are processing
* spec tooling to support bulk upload multi year
* row parser now has year in namespacing
* add static data for 2023 row parser
* add placeholder to log to csv for specs
* bulk upload aribtrary 23/34 column ordering works
* bulk upload supports 23/24 without headers
* bulk upload 23/24 supports BOM + invalid chars
* dupe tests
* port 23/24 attributes_for_log
* port 23/24 bulk upload validations
* force crossover period
* tweak max permitted columns
* able to return column for given field
* work out column for field for errors
* add field_4 as 23/24 setup field
* remove duplicate method
* map schemes and locations correctly
* handle arbitrary number of header rows
* add missing fields to bulk upload support
* fix bulk upload age data types
* bulk upload handles both with and without headers
- headers are from the spreadsheet template
- otherwise assume cell A1 is start of the dataset
* add class to create logs from bulk upload
* create logs when processing bulk uploads
* remove bulk_upload_id from csv output
* create bulk upload logs only if all valid
- this will be changed later to allow for partial logs
- and only to create logs when a threshold has been met
* add method to blank invalid non setup fields
* bulk upload log creation blanks invalid fields
* fix incorrect logic for bulk upload renewal
* fix linting
* bulk upload log creation fail logs to sentry
* fix bulk upload line ending parsing
* extract bulk uploading csv parsing to class
* use csv parser in log creator
* change handle line endings mechanism
- we now strip all windows line endings for unix based line endings
- this normalises things making it simpler
Rachael Booth
kosiakkatrina
natdeanlewissoftwire
Jack
Arthur Campbell
Phil Lee