# Context
- https://digital.dclg.gov.uk/jira/browse/CLDC-2279
# Changes
- split bulk upload feature toggle by log type ie lettings and sales
- this allows us to toggle with finer granularity
- moved `FeatureToggle` from initializer to service class. not sure why it was an initializer in the first place? this means its available to zeitwerk and will be reloaded when needed
* feat: update scheme status so incomplete unless has active locations
* feat: update scheme typeahead text
* feat: reject incomplete locations
* feat: show completed/incomplete locations and validate when completed == 0
* feat: improve copy
* feat: change active -> completed
* feat: update scheme typeahead text
* feat: reject incomplete locations
* feat: show completed/incomplete locations and validate when completed == 0
* feat: improve copy
* feat: change to confirmed to add clarification in code
* feat: update scheme typeahead text
* feat: reject incomplete locations
* feat: show completed/incomplete locations and validate when completed == 0
* feat: improve copy
* feat: update scheme typeahead text
* feat: reject incomplete locations
* feat: only confirm locations if complete (old) AND save button clicked (new)
* feat: add unconfirmed scope
* refactor: complete -> confirm
* feat: fix tests
* feat: update scheme typeahead text
* feat: reject incomplete locations
* feat: show completed/incomplete locations and validate when completed == 0
* feat: improve copy
* feat: update scheme typeahead text
* feat: reject incomplete locations
* feat: only confirm locations if complete (old) AND save button clicked (new)
* feat: add unconfirmed scope
* refactor: complete -> confirm
* feat: fix tests
* feat: fix more tests
* feat: fix more tests
* refactor: rubocop
* refactor: rubocop
* refactor: rubocop
* feat: add tests for incomplete schemes
* refactor: linting
* feat: test incomplete count too
* feat: test unconfirmed locations aren't options in location_id
* feat: test validation
* feat: test when scheme is confirmed
* feat: test when scheme is confirmed for support user
* feat: test when location is confirmed
* refactor: linting
* refactor: consistent use of factorybot or not within a file
* feat: confirm locations when complete even if save button not clicked, and update tests
* refactor: simplify
* refactor: simplify
* refactor: move location helper methods to helpers
* refactor: move scheme helper methods to helpers
* refactor: formatting
* refactor: po response
* refactor: review response
* add in a new option for the period question and reorder existing answer options
* ensure that the lettings log calculates values correctly that relate to the period
* ensure that depends on conditions related to period take account of the new option
relatedly create methods on LettingsLog to allow improved readability in these depends on conditions
refactor depends on conditions with these new method
* update the 22_23 json form to use the new option as well
* update tests to cover deriving variables that depend on period
update test fixture to allow this
* refactor some depends on and amend LEttingsLog method to improve readability
* remove forgotten focus in spec file found in linting check
* amend context descriptino in new tests
* refactor a couple more depends on for readability
* rename variables in test file
* write tests for new validations
* validate that answers to numeric questions are given to the correct level of accuracy and in normal format with only digits
* add error messages for new validations
* add or alter step on numeric questions in sales, amend one test given step changes
* copy change
* remove validation that is specific to number format and fold that functionality into accuracy validation, rejig tests accordingly
* fix various tests
* remove reference to currency in validation message that concerns step only and doesn't check whether firled is currency related
* alter copy to avoid redundant repetition in validaiton message
* save changes made in rebase
* changes after PO review
two step values had been missed in previous work
in various places there were custom validations applied that duplicated the functionality of the generic min max validation that is applied to all numeric questions, in these situations sometimes the min and max on the question class was inconsistent with the limit that triggered validations
these have been corrected and made consistent
various tests were affected by this and have been amended accordingly
* remove tests introduced by mistake in rebase
* Move and rename some tests
* Add a merge organisation link
* Add merge page
* Don't display a second nav bar for support
* Fix spacing
* fix spacing and typos
* Allow interruption screens to format money values
* Remove formatting from saleslog method
* Use money flag
* Use pre exhisting method to format values
* Update config/locales/en.yml
Co-authored-by: Arthur Campbell <51094020+arfacamble@users.noreply.github.com>
---------
Co-authored-by: Arthur Campbell <51094020+arfacamble@users.noreply.github.com>
* Format errors on money amounts with 2 decimals
* CLDC-1975 Format money input and error messages
* format discounted_ownership_value
* Format more locales
* Update lettings_log tests
* Fix import service spec
* create method to test whether we are currently in the crossover period and associated tests
* update copy, use method for testing whether we are in the crossover period, remove focus from test file
* reuse existing method to determine whether we are in a collection period
* use the existing method and update validations
* fix test broken by changes
* update location in same way as scheme
* create method on FormHandler that finds the start date of the earliest collection period
* ensure that default deactivation and reactivation dates also reflect the changes
* create tests for the new validations
* lint correction
* minor copy change
* minor logic change
* amend naming error after rebase conflict
remove test that is no longer correct after change on another branch to functionality
update a test after a copy change
* add first page for bulk upload resume journey
* bulk upload resume handles upload again
* add confirm page to bulk upload resume journey
* replace placeholder count with correct value
* apply recommendation for bulk upload resume choice
* add how to fix bulk upload mailer
* integrate new bulk upload approve journey
* add missing bulk upload error mappings
* remove test
* prevent approve being called multiple times
* bulk upload creates invisible logs ahead of time
* work invisible logs into bulk upload flow
* sort errors so deterministic
* remove unused ensure
* remove expected_log_count and processed
- these fields are no longer used or needed
* introduce pending status
* swap visible for pending logs
* only show visible lettings logs
* hard code status filters
* remove unused model methods
* only show visible sales logs
* form controller ignores hidden logs
* locations and schemes only affect visible logs
* add a validation against the case where monthly leashold charges are zero, test this, remove unnecessary validation on cash discount as this is covered by validate_numeric_min_max
* remove focus: true from describe block
* minor changes for testing and linting
* correct minor bug found in PO
* make minor amendments to failing tests
* feat: add void date guidance partial
* feat: set guidance pos bottom for 22/23 log
* feat: add tests
* refactor: linting
* Update app/models/form/question.rb
Co-authored-by: Phil Lee <asmega@users.noreply.github.com>
* feat: smart quotes
* feat: respond to po comments
* feat: add voiddate/mrcdate validation to mrcdate too
* feat: add ten year validations to startdate too
* feat: fix tests
---------
Co-authored-by: Phil Lee <asmega@users.noreply.github.com>
* Clear import fields failing non_temp_accommodation validation
* Clear joint tenancy if there is only 1 person in the household
* Clear offered if the value is above the maximum of 20
* Display error message on ecstat
* Remove ecstat if the income is too high for it
* Update service charge validations
* Create location deactivation periods instead of setting a scheme end date
* Add test for deactivation period
* Update ranges in tests
* Remove scheme validation test
* Check that the location gets confirmed
---------
Co-authored-by: Kat <katrina@kosiak.co.uk>
* add method to sales log and use in depends on to increase readability
* increase minimum allowed value for mortgage
* write tests and add validation error to config for new validation
* write validation
* adjust validation so it doesn't trigger when mortgage value is derived as 0
* correct linting problem and update tests to account for change in validation
* Add carehome charges value check field
* Add carehome charges soft validation to 23/24 forms
* Update care_home_charge_expected_not_provided?
* Add care_home_charges_value_check to 22/23
* Update csv
* Add care_home_charges_value_check to 21/22 form
* Update import to set value_chech to yes
* add some methods on lettings log and use them and already existing methods to make depends on human readbale for pages in property information subsection
* reformat answer options for neatness
* ensure that property information subsection is conditionally not shown under correct conditions
* amend method to find the path for the next question to answer so that if no questions in the next section are currently routed to, it goes to the section after that
* make amendments for failing test and linting
* remove unnecessary method
* remove unnecessary conditions from depends on, the displayed in tasklist method override is sufficient
* test the change to the method creating a link to the next subsection
* minor refactor to displayed_is_tasklist condition and changes in testing
* implement the fix on 22/23 form also
* write a test covering the fix
* removal not_renewal? method and tweak depends on for all pages using that method
* linty fluff begone
* correct some errors in tests after changes to factories
* minor testing amendments after rebase
* Add discounted_sale_value_check to sales
* Add DiscountedSaleValueCheck pages and questions
* Add value check to the form
* Only run validate_discounted_ownership_value as a hard validation on and after 24/25
* Rebase fix
* Fix form spec
* Update discounted_ownership_value_invalid?
* update validation
* Update soft validation content
* PR comments
* typo
* lint
* Default discounted_sale_value_check on import
* Add local authority links
* Display all local authorities for location
* set correct LA for log based on year
* Format local authorities for locations
* Rename variable
* Add import task for la links
* look at form start date because log start date might not be given
* Update app/models/lettings_log.rb
Co-authored-by: James Rose <james@jbpr.net>
* Update app/helpers/locations_helper.rb
Co-authored-by: James Rose <james@jbpr.net>
* Refactor
* Seed review app
* Change dates format
* Update the local authority link data
* Typo
* update mapping in Lettings logs
---------
Co-authored-by: James Rose <james@jbpr.net>
* feat: add validation that staircase % is >= threshold
* refactor: dry out validation
* refactor: correctly order shared ownership types in array
* test: add tests for stairbought threshold validation
* chore: lint
* feat: all staircasing transactions must have stairbought >= 1%
* feat: add validation to shared ownership type as well
* chore: lint
* refactor: simplify no errors tests
* refactor: simplify tests by using iteration
* feat: skip fixed term tenancy length question when tenancy is other
* CLDC-2006 Add 23/24 collection year LAs (#1392)
* Update LAs for 23/24 sales questions
* Update LAs for 23/24 lettings questions
* Update LAs for 23/24 location
* Remove redundant assignments
* Update question files to use the local authorities from db
* Update locations to use local authorities from db
* Rebase changes
* Replace hardcoded date
* Add PaperTrail to OrganisationRelationship (#1406)
* CLDC-1838 update copy for both offered questions (#1401)
* ensure copy is changed for both questions relating to how many times a property has been offered
extract to en.yml so that both questions keep identical copy in the future
* minor typo caught in code review
* copy change for LA (#1407)
* CLDC-1997 Infer new LAs for post 2023 logs (#1398)
* Infer new LAs after year 2023
* Test
* Run CSV export before XML export to CDS (#1393)
- The XML export mutates the `LogsExport` table which determines the period in which logs are exported.
- The CSV export does not change this table, so if the CSV gets ran at the same time as the XML log it could skip records.
* CLDC-1998 Correctly infer 22/23 las if a NEW one is returned by postcodes.io (#1397)
* feat: correctly infer las for 22/23 logs
* feat: copy changes to sales log
* feat: don't reset las when clash with postcode to make tests pass
* feat: align with similar PR for 23/24
* feat: tidying and add test
* refactor: lint
* refactor: use unless
* CLDC-1919 Add 23/24 rent ranges (#1405)
* feat: add 23/24 rent ranges csv
* fix: use new 2023/24 rent ranges after duplicates removed
* test: update seeds test to use new rent ranges count
* fix: resave 2023 CSV as not UTF-8 (#1411)
* CLDC-1820 Update reason for leaving domestic abuse answer options (non-renewal only) (#1391)
* feat: change domestic abuse answer opts
* test: add test for reason question
* CLDC-1808 add previous tenure question for buyer two (#1354)
* create and run migration for new columns related to buyer two previous tenure
* a little refactoring for readability
* create new pages, questions, associated tests and a couple of boolean methods on sales log for depends_on readability
* add new pages to subsection and amend tests to reflect this change
* make new pages only visible for new collection year, rename filesamend tests
* some copy changes and alterations of readability after code review
* CLDC-1816 add hint text to tenancy type options (#1341)
* add hint text to several options on tenancy type, rename question class and a few others to make them pascal case and write test files for all changed questions
* update some depends on for readability and write test files for page classes
* change the condition to avoid conflicts with 2034 and cover that ticket in this one
* also add hints to other tenancy question
* add new answer options to nationality, rename question, write test file (#1339)
* CLDC-2014 remove text field for buyer 1 nationality other (#1388)
* remove question no longer needed and associated test file
* amend page and the other question in that page that determined whether the deleted question should be shown
update associated test files
* createa a migration to remove the column for the deleted question
* update test broken by changes
* fix test broken by changes
* CLDC-1886 Add sales paper form download (#1408)
* Add sales paper form download
* Rename download path
* Add 22/23 sales log download
* Update lettings log form download (#1409)
* Run tests in parallel (#1412)
* Run tests in parallel
* Failing test
* Add parallel tests to prod pipeline
* Remove failing test
* Add PARALLEL_TEST_PROCESSORS env var
* CLDC-1925 ask for buyer 2 nationality when joint purchase (#1390)
* create migration for new column
* create page class and add to household characteristics
* create question for buyer 2 nationality and associated test file
* fix failing test after added page
* rename method after code review
* add question number for the new question to allow CLDC-1864 to progress with fewer conflicts/complications
* fix: get real validation messages for rent range validations (#1413)
* CLDC-1826 lettings log codes only download (#1268)
* update seeds to add self in review env, change spec to reflect this, update config yml to allow csv exports in review
* update interface of relevant methods
EmailCsvJob, LettingsLog.to_csv and LettingsLogCsvService consume codes_only flag
* update tests including adding a new csv file to test against
* update LettingsLogCsvService to output codes only csv
* correct minor error and linting
* enable codes only download in UI
- add link on lettings log index page
- pass codes_only flag through params in relevant links and methods
- convert flag to boolean in controller methods
* ensure link displayed successfully for all renderings of logs_list and params passed through relevant methods in organisations controller
* fix existing tests
* correct linting thing
* correct linting error
* update tests for lettings log controller
* correct linting errors
* update organisations controller tests
* make minor changes after code review
* remove changes made for testing on review app
* make codes only download visible to support users only
* change variable names throughout after info on rauby/rails naming conventions, update tests for change in who can view codes only download link
* rework csv service for readability, remove delegating methods from lettings log to keep all code to do with mapping between our domain and desired export format in one place
* update test name
* correct a small typo and remove a duplicated method after clever git merge conflict suggestion
* point review app at staging csv bucket for csv download
* change variables named codes_only_export to codes_only to avoid inconsistency
* write tests to ensure that differetn user roles have the correct permissions around csv download
* ensure that non support users may not download codes only exports
* correct a small error in a previous commit
* correct minor linting error
* Make tests less dependant (#1415)
* Seed the test db before tests and reset forms after tests
* Update rails
* Remove redundant FormHandler reinitialise
---------
Co-authored-by: kosiakkatrina <54268893+kosiakkatrina@users.noreply.github.com>
Co-authored-by: James Rose <james@jbpr.net>
Co-authored-by: Arthur Campbell <51094020+arfacamble@users.noreply.github.com>
Co-authored-by: Phil Lee <asmega@users.noreply.github.com>
Co-authored-by: SamSeed-Softwire <63662292+SamSeed-Softwire@users.noreply.github.com>
* refactor questions from validator to row parser
* able to switch between bulk upload parsers
- depending on what year we are processing
* spec tooling to support bulk upload multi year
* row parser now has year in namespacing
* add static data for 2023 row parser
* add placeholder to log to csv for specs
* bulk upload aribtrary 23/34 column ordering works
* bulk upload supports 23/24 without headers
* bulk upload 23/24 supports BOM + invalid chars
* dupe tests
* port 23/24 attributes_for_log
* port 23/24 bulk upload validations
* force crossover period
* tweak max permitted columns
* able to return column for given field
* work out column for field for errors
* add field_4 as 23/24 setup field
* remove duplicate method
* map schemes and locations correctly
* handle arbitrary number of header rows
* add missing fields to bulk upload support
Phil Lee
James Rose
Arthur Campbell
natdeanlewissoftwire
kosiakkatrina
Jack
SamSeed-Softwire