# Context
- https://digital.dclg.gov.uk/jira/browse/CLDC-2287
- bulk upload errors on setup fields are not being categorised as setup errors
# Changes
- ensure errors on setup field are categorised as `setup` errors
- ordering of validations tweaked, so `validate_nulls` is further down the execution order. this is so any existing validation run first and `validate_nulls` only adds errors if there aren't any existing errors on a field
- removed old 16/60% tests are no longer required due to introduction of `how to fix` journey
* Clear invalid format previous postcode
* Fix import with income 2 outside of london validation
* Remove mscharge if it is under minimum
* Clear frombeds if it's out of range
* create a method on the FormHandler that returns the sales form questions for all years in the order that they appear in the form
* update csv email job to accomodate sales log export as well as lettings
add to tests to reflec the changes made
* write tests to cover the desired functionality of the SalesLogCsvService
* create the SalesLogCsvService
create a necessary method on the log to enable submission method to be included on the csv
derive values for the two halves of previous postcode for export
* add relevant links in the UI and pipe everything together in controllers
amend organisations controller to have flexibility to download logs of either type
add necessary methods to sales log controller, raising shared method to logs controller
update routing for amendments and additions
extract helper method to build urls for downloading logs within an organisation
* correct various linter complaints and tech review suggestions
* minor amendment to add old_id and reorder early columns
* undo my 'clever' refactor that broke things
* refactoring of csv service after some tech review and some UI testing in review app
* update tests to include a test of a full export and all values in teh csv
* correct minor routing error to ensure correct url is shown and tab selected after requesting csv email
* update organisations controller requests spec file to cover new functionality and make a minor amendment to authentication scope in the controller after error found in testing
* write request tests for the new functionality in the sales log controller, define authorisation in the controller
* minor correction after rubocop's kind suggestion'
* various corrections from first pass at PO, tech review, linter, etc
* refactor :ordered_sales_questions_for_all_years
* first pass at implementing flexible code-based form fixtures for testing
* second pass
* refactor all tests of :ordered_sales_questions_for_all_years to use new factories
* some refactoring in the testing of the csv service
* use that fact that params is always available in controllers and don't pass it around, inline some methods calls
* correct minor bug to ensure that "Return to logs" link returns to the correct index page
* remove reminder comments
* write further tests on the manipulation of questions into the csv headers, update factories of form constituents to allow the creation of forms with richer questions
* fix linter complaints
* minor alterations after rebase to account for changes made on other branches
* refactor after code review
* tweak fixtures after rebase containing alterations to the factory defaults
* Clear all the charges if the error is on tcharge
* Round earnings value upon import
* Rounds savings to the nearest 10
* Clear income on over_hard_max_for_london validation
* Refactor sale import validations to be consistent with lettings
* Extract charges attributes into a variable
# Context
- Removing dependencies that are not used
# Changes
- Remove chart.js which appears to be part of activeadmin and chartkick which seemed to have been removed
* Add affected_question_ids to pregnancy check
* Update is_referrer_interruption_screen? check and naming
* Use interruption_screen_question_ids to set soft validation errors on relevant fields
* Add soft validations to sales bulk upload
* Add soft validations to lettings logs 23/24 bulk upload
* Add errors for optional soft validations
* Only add soft validations once
* Import helper methods
* Update test based on new validation messages
* Rebase fix
* add a validation to prevent an inconsistent combination of values and tests for this validation
* show related method in diff
* remove comment
* extract reusable logic from SalesLogVariables to independent module
* update lettings log tests around derivations related to renewal
* refactor derivation logic to share functionality with sales logs where possible
* remove some tests which are now duplicated above, refactor tests using Jack's wonderful :change suggestion, add in tests about logic deriving vacdays that was not previously covered
* minor changes after tech review
needstype is a field that contributes to the vcalidation and can therefore trigger the validation
relevant test file has also been updated in line with this change
* Add new organisation address fields to merge requests
* Add new organisation address page
* Add new organisation telephone number placeholder page
* close the div
* Add new organisation name column
* Update new organisation name page
* Refactor validate_response
* Add placeholder new_organisation_address page
* Add existing organisation name validation
* Refactor error messages
* Extract some validations to the model
* Update which field we add the errors to
* add several methods to the sales log to allow subsequent work to be human readable
* add a validation to prevent an inconsistent combination of values and tests for this validation
* handle derivation of values around buyers living in the property
- derive values where appropriate
- clear these values when the derived state no longer holds
- update the routing for pages holding questions about whether particular buyers will live in the property to reflect when they are derived
- test the deriving and associated clearing of values
- update tests on page routing and sales log factory
* update a page routing condition for human readability using an existing method and update test to reflect this change
* show related method in diff
* minor amendments after tech review
* simplify reset_derived_questions after tech review
* refactor on deriving and clearing invalid derived values on sales log
* correct linter complaints
* remove comment
* add validation to one more field with a new error message as it is in fact possible to tirgger the validation in the setup section
# Context
- https://digital.dclg.gov.uk/jira/browse/CLDC-2294
- When bulk upload returns a results page it would be useful if colleagues of the uploader can see this page to help fix errors
- It would also be useful if support users can see these reports to help diagnose bulk upload errors
# Changes
- Added `pundit` gem to handle authorization
- Bulk upload results previously only accessible to the bulk uploader. Now they can be seen by users in the same org as the uploader and also support users
# Context
- Partially related to https://digital.dclg.gov.uk/jira/browse/CLDC-2316
- Comprehending sales or lettings bulk upload CSV is enough, comprehending both simultaneously is rather challenging
# Changes
- Split out test helper class by log type ie lettings/sales
# Context
- https://digital.dclg.gov.uk/jira/browse/CLDC-2301
- Content for 2022 and 2023 bulk upload journey need to differ to give users the correct info they need to complete the journey
# Changes
- Update 2022 and 2023 `Prepare your file` and guidance pages
* [CLDC-2202] Allow coordinators to set created_by
* Scope user selection when data coordinator
* Remove managing_for_other_user_enabled
* Move sales created_by page and question out of common
* Address comments
- only select required users
- remove not needed CYA checks
* CLDC-2056 Add absorbing organisation question
* Handle next page and previous_template with `page`
* Hardcode backlinks
* Refactor flow to store value when selecting new org option
* Handle unanswered question
* Update error copy
# Context
- https://digital.dclg.gov.uk/jira/browse/CLDC-1888
- This is a continuation of https://github.com/communitiesuk/submit-social-housing-lettings-and-sales-data/pull/1277
- When bulk uploading we want to check users are not uploading data that already exists to prevent them submitting duplicate
# Changes
- This feature is behind a feature toggle. it has been disabled for staging for testing purposes but available in all other environments
- If a log already exists based off certain fields add errors to the associated fields
- We discount any hidden logs and only check "active" logs
- Added memoization to `#valid?` as an optimisation
* feat: add validation for number of field labels
* feat: check field labels are numbers
* feat: remove validations for number of columns
* refactor: rename non_blank_fields_count to valid_field_numbers_count
* test: add functions to generate custom field labels/values
* test: that extra invalid field labels don't cause issues
* test: that removing a valid field label reduces count by 1
* test: that adding a valid field label increases count by 1
* refactor: rename validate_fields_count and wrong_field_count
* feat: add validation that max col count not exceeded when no headers
* fix: convert numbers to strings in default_field_numbers
* feat: add leniency to max cols count (in case of 1 extra col)
* test: explicitly set year of bulk upload to be 2022
* test: add/update tests in validator_spec
* chore: lint
* test: remove tests from csv_parser_spec that were moved to validator_spec
* feat: update 2022 csv_parser to work with new validations
* refactor: define number of valid 2022 fields in one place
* refactor: remove redundant headers definition
* feat: update 2022 csv parser to have col flexibility like 2023
* test: for validator 2022 as well as 2023
* test: simplify 2022/2023 logic and improve layout
* chore: lint
* test: ensure context descriptions start with "when"
* refactor: check fields/columns count within csv parser, not validator
* test: update 2022 csv parser tests to work like 2023
* chore: lint
Phil Lee
Jack
kosiakkatrina
Arthur Campbell
dependabot[bot]
James Rose
SamSeed-Softwire