* some minor refactoring
remove methods from child class that replicate methods on the parent class
tidy up check for nil
remove gubbins and inline method body given only used once
* update import services for lettings and sales to import creation method
write tests to cover this
* create sales log field import service and associated spec file, with methods and tests for importing the creation method of logs that have already been imported
* update lettings log field import service and related spec to allow importing creation method of logs
* use the methods dynamically created by active record in all relevant places, removing obsolete methods in teh process.
various tests tweaked to suppor this change.
rake task from another ticket folded into this ticket to prevent merge conflicts
* rename method for ruby conventions
* update PR for altered spec
upload id now decided to be a better indicator of bulk upload status, import service amended accordingly
tests updated in line with this
* update field import services in line with import services to use upload id rather than upload method as the source of truth for how a log was created
* slight refactor to reduce nesting and dodge linter complaints
* minor amendment to log creator spec in bulk upload to use enum dynamic methods
* form handler to return all questions from lettings forms for all years with ordering interleaved
functionality and tests
* refactor lettings log csv service and all associated tests
remove methods on log models when we can call them directly on associated models
update job to call the service directly with the collection of logs rather
minor modifications to the sales log csv service
update many test files to test the appropriate logic in the appropriate place
* tidying
final amendments to tests
remove commented code
rename variable
* change the position of the rent value check field in the headers
* CLDC-2492 add creation method field to logs (#1738)
* create migrations to add creation method fields to both log types
* add enum definition to logs for creation method
* upadte csv export services to retrieve creation method values direct from the log, remove methods previously used from the log model
* run migrations to update schema
* ensure that logs created via bulk upload have this set correctly when created
* feat: setup to replicate failures to fix
* feat: wip test fixes
* feat: remainder of current state test fixes
* feat: set form date to past to trigger errors to fix
* feat: revert, don't want this in final diff
* feat: fix row_parser tests
* feat: sales log importer and validator fixes
* feat: remainder of test fixes
* refactor: lint
* Remove soft validations from sales setup and make them mandatory
* Remove soft validations from lettings setup and make them mandatory
* Fix remaining tests
* Add card numbers to pregnancy checks
* call form to reset invalid values when saving a sales log
* fix tests failing after updating sales log functionality to reset invalid answers
* add a test to cover the new functionality added
* a further test
* create a method on the FormHandler that returns the sales form questions for all years in the order that they appear in the form
* update csv email job to accomodate sales log export as well as lettings
add to tests to reflec the changes made
* write tests to cover the desired functionality of the SalesLogCsvService
* create the SalesLogCsvService
create a necessary method on the log to enable submission method to be included on the csv
derive values for the two halves of previous postcode for export
* add relevant links in the UI and pipe everything together in controllers
amend organisations controller to have flexibility to download logs of either type
add necessary methods to sales log controller, raising shared method to logs controller
update routing for amendments and additions
extract helper method to build urls for downloading logs within an organisation
* correct various linter complaints and tech review suggestions
* minor amendment to add old_id and reorder early columns
* undo my 'clever' refactor that broke things
* refactoring of csv service after some tech review and some UI testing in review app
* update tests to include a test of a full export and all values in teh csv
* correct minor routing error to ensure correct url is shown and tab selected after requesting csv email
* update organisations controller requests spec file to cover new functionality and make a minor amendment to authentication scope in the controller after error found in testing
* write request tests for the new functionality in the sales log controller, define authorisation in the controller
* minor correction after rubocop's kind suggestion'
* various corrections from first pass at PO, tech review, linter, etc
* refactor :ordered_sales_questions_for_all_years
* first pass at implementing flexible code-based form fixtures for testing
* second pass
* refactor all tests of :ordered_sales_questions_for_all_years to use new factories
* some refactoring in the testing of the csv service
* use that fact that params is always available in controllers and don't pass it around, inline some methods calls
* correct minor bug to ensure that "Return to logs" link returns to the correct index page
* remove reminder comments
* write further tests on the manipulation of questions into the csv headers, update factories of form constituents to allow the creation of forms with richer questions
* fix linter complaints
* minor alterations after rebase to account for changes made on other branches
* refactor after code review
* tweak fixtures after rebase containing alterations to the factory defaults
* feat: unrelated currency formatting
* Revert "feat: unrelated currency formatting"
This reverts commit 75bd3efbf8.
* feat: add soft val pages to property information section (not to setup)
* feat: update tests
* feat: update tests
* feat: add validation to setpu section
* refactor: lint
* test: update
* feat: set nil as default check answers card no
* feat: use : to define more explicitly
* feat: make optional as affects setup
* feat: update tests
* feat: put la in nil card with rest of uprn/address qs
* feat: route to uprn confirmation from check answers changes to uprn even though not an interruption screen
* feat: update tests
* feat: update tests
* refactor: lint
* feat: update tests
* feat: update tests
* feat: wip blank fields and dependent fields on upload tos ee if valid and can upload with missing info - this is not the exact ac on the ticket yet
* Revert "feat: wip blank fields and dependent fields on upload tos ee if valid and can upload with missing info - this is not the exact ac on the ticket yet"
This reverts commit 136e5f6f12.
* feat: fix uprn hidden in check answers
* feat: fix tests
* feat: fix tests
* rename variables in test file
* write tests for new validations
* validate that answers to numeric questions are given to the correct level of accuracy and in normal format with only digits
* add error messages for new validations
* add or alter step on numeric questions in sales, amend one test given step changes
* copy change
* remove validation that is specific to number format and fold that functionality into accuracy validation, rejig tests accordingly
* fix various tests
* remove reference to currency in validation message that concerns step only and doesn't check whether firled is currency related
* alter copy to avoid redundant repetition in validaiton message
* save changes made in rebase
* changes after PO review
two step values had been missed in previous work
in various places there were custom validations applied that duplicated the functionality of the generic min max validation that is applied to all numeric questions, in these situations sometimes the min and max on the question class was inconsistent with the limit that triggered validations
these have been corrected and made consistent
various tests were affected by this and have been amended accordingly
* remove tests introduced by mistake in rebase
* amend step value on earnings
* write tests that invalid answers are cleared when the start date of a log is changed
* clear invalid answers when the start date of a log is changed
* correct error
* undo previous work to take new direction
* write tests to cover updated functionality of method in form to reset values when they are not routed to or when answers to radio questions are no longer valid
* update method in form to improve readability and to ensure that when the answers to radio questions are no longer valid, taht they are cleared
* add back an "end" deleted by mistake and fix a linting issue
* make some minor copy and variable name corrections
* fix a broken test after updates
* remove bulk upload tests for adding validations to unpermitted values to radio questions, these are now being cleared before validation
* fix some tests broken after rebasing
* add step to age questions in sales
* remove list of question ids from the form that should not be cleared, this information should be held on the questions themselves
* rename variables
* rename instance variable to be nice and long
* minor changes after rebase
* add in a new option for the period question and reorder existing answer options
* ensure that the lettings log calculates values correctly that relate to the period
* ensure that depends on conditions related to period take account of the new option
relatedly create methods on LettingsLog to allow improved readability in these depends on conditions
refactor depends on conditions with these new method
* update the 22_23 json form to use the new option as well
* update tests to cover deriving variables that depend on period
update test fixture to allow this
* refactor some depends on and amend LEttingsLog method to improve readability
* remove forgotten focus in spec file found in linting check
* amend context descriptino in new tests
* refactor a couple more depends on for readability
* rename variables in test file
* write tests for new validations
* validate that answers to numeric questions are given to the correct level of accuracy and in normal format with only digits
* add error messages for new validations
* add or alter step on numeric questions in sales, amend one test given step changes
* copy change
* remove validation that is specific to number format and fold that functionality into accuracy validation, rejig tests accordingly
* fix various tests
* remove reference to currency in validation message that concerns step only and doesn't check whether firled is currency related
* alter copy to avoid redundant repetition in validaiton message
* save changes made in rebase
* changes after PO review
two step values had been missed in previous work
in various places there were custom validations applied that duplicated the functionality of the generic min max validation that is applied to all numeric questions, in these situations sometimes the min and max on the question class was inconsistent with the limit that triggered validations
these have been corrected and made consistent
various tests were affected by this and have been amended accordingly
* remove tests introduced by mistake in rebase
* add first page for bulk upload resume journey
* bulk upload resume handles upload again
* add confirm page to bulk upload resume journey
* replace placeholder count with correct value
* apply recommendation for bulk upload resume choice
* add how to fix bulk upload mailer
* integrate new bulk upload approve journey
* add missing bulk upload error mappings
* remove test
* prevent approve being called multiple times
* bulk upload creates invisible logs ahead of time
* work invisible logs into bulk upload flow
* sort errors so deterministic
* remove unused ensure
* remove expected_log_count and processed
- these fields are no longer used or needed
* introduce pending status
* swap visible for pending logs
* only show visible lettings logs
* hard code status filters
* remove unused model methods
* only show visible sales logs
* form controller ignores hidden logs
* locations and schemes only affect visible logs
* Fix possible bulk upload years
* Initialise 2021/22 form routes for tests
* Return test routes from form handler
* Fix some tests
* fix bulk upload tests
- change from mocks to actual code calls
* update years for bulk upload form
* Fix some uprn tests
* Set the scheme as available from previous year if the collection is in crossover
* Update location available_from
* Update location and scheme tests
* Refactor and comment out a scheme test
* Some uprn and export tests
* Fix lettings logs controller tests and comment out previous collection message test
* fix tests for new collection year
* fix bulk upload tests
for some reason tenancylength must not be given for validation to pass
* update bulk upload tests for new year
* More fixes
* Undo location changes
* Fix remaining uprn
* further new collection year test fixes
---------
Co-authored-by: Phil Lee <asmega@users.noreply.github.com>
* add a validation against the case where monthly leashold charges are zero, test this, remove unnecessary validation on cash discount as this is covered by validate_numeric_min_max
* remove focus: true from describe block
* minor changes for testing and linting
* correct minor bug found in PO
* make minor amendments to failing tests
* Clear import fields failing non_temp_accommodation validation
* Clear joint tenancy if there is only 1 person in the household
* Clear offered if the value is above the maximum of 20
* Display error message on ecstat
* Remove ecstat if the income is too high for it
* Add carehome charges value check field
* Add carehome charges soft validation to 23/24 forms
* Update care_home_charge_expected_not_provided?
* Add care_home_charges_value_check to 22/23
* Update csv
* Add care_home_charges_value_check to 21/22 form
* Update import to set value_chech to yes
* add some methods on lettings log and use them and already existing methods to make depends on human readbale for pages in property information subsection
* reformat answer options for neatness
* ensure that property information subsection is conditionally not shown under correct conditions
* amend method to find the path for the next question to answer so that if no questions in the next section are currently routed to, it goes to the section after that
* make amendments for failing test and linting
* remove unnecessary method
* remove unnecessary conditions from depends on, the displayed in tasklist method override is sufficient
* test the change to the method creating a link to the next subsection
* minor refactor to displayed_is_tasklist condition and changes in testing
* implement the fix on 22/23 form also
* write a test covering the fix
* removal not_renewal? method and tweak depends on for all pages using that method
* linty fluff begone
* correct some errors in tests after changes to factories
* minor testing amendments after rebase
* Add local authority links
* Display all local authorities for location
* set correct LA for log based on year
* Format local authorities for locations
* Rename variable
* Add import task for la links
* look at form start date because log start date might not be given
* Update app/models/lettings_log.rb
Co-authored-by: James Rose <james@jbpr.net>
* Update app/helpers/locations_helper.rb
Co-authored-by: James Rose <james@jbpr.net>
* Refactor
* Seed review app
* Change dates format
* Update the local authority link data
* Typo
* update mapping in Lettings logs
---------
Co-authored-by: James Rose <james@jbpr.net>
* update seeds to add self in review env, change spec to reflect this, update config yml to allow csv exports in review
* update interface of relevant methods
EmailCsvJob, LettingsLog.to_csv and LettingsLogCsvService consume codes_only flag
* update tests including adding a new csv file to test against
* update LettingsLogCsvService to output codes only csv
* correct minor error and linting
* enable codes only download in UI
- add link on lettings log index page
- pass codes_only flag through params in relevant links and methods
- convert flag to boolean in controller methods
* ensure link displayed successfully for all renderings of logs_list and params passed through relevant methods in organisations controller
* fix existing tests
* correct linting thing
* correct linting error
* update tests for lettings log controller
* correct linting errors
* update organisations controller tests
* make minor changes after code review
* remove changes made for testing on review app
* make codes only download visible to support users only
* change variable names throughout after info on rauby/rails naming conventions, update tests for change in who can view codes only download link
* rework csv service for readability, remove delegating methods from lettings log to keep all code to do with mapping between our domain and desired export format in one place
* update test name
* correct a small typo and remove a duplicated method after clever git merge conflict suggestion
* point review app at staging csv bucket for csv download
* change variables named codes_only_export to codes_only to avoid inconsistency
* write tests to ensure that differetn user roles have the correct permissions around csv download
* ensure that non support users may not download codes only exports
* correct a small error in a previous commit
* correct minor linting error
* Add local authorities table
* Seed initial local authorities
* set previous_location_only to false by default
* Add unique index and rake task
* Update field name and required fields
* Remove previous_location_only field
* tests
* Only process sales logs in sales import job
* Only process lettings logs in lettings import job
* Add sales log import task to the full import
* change review app import bucket name
* Update mortgageused and joinmore
* Update jointmore and ownership scheme
* Fix ownershipsch so that it doesn't override to nil
* Set default relat2, update default household count
* het ownership from type if not given
* Improve logging
* Remove mortgageused method
* Look at Q16aProplensec2 column if Q16aProplen2 is empty
* Set default income used and pregblank
* Remove fields calculated internally
* Comment out sales import from full import task
* typo and change bucket name
* Create test fixtures
* Add old id column
* Add sales logs importer
* Save and update completed discounted ownership log without postcode
* Update fixtures to be incomplete
* Import a completed shared ownership example
* Test with a non homebuy shared ownership example
* Add privacynotice
* Update hholdcount and confirmed fields
* Add buyer stilll serving mapping
* Add totadult/totchild and outright sale examples
* Update log fixtures
* Extract shared methods into logs import service and lint
* Add sales logs import rake task
* Update noint, xml examples and some mappings
* Add tests for checking that all required questions are answered
* Update tests, clean up import
* Map mortgage lender and mortgage lender other
* Infer Mscharge known as no for outright sale
* refactor setting default values
* when the armedforcesspouse is not answered set is as don't know
* Refactor tests: change log id names
* set savings to not known if not given
* Refactor tests: change nesting
* Backfill default household characteristics for completed log
* Add more default mapping
* Typo
* Improve logging and refactor tests
* Adjust test data to fit with the mappings that are known so far
* Rename fixture files
* add class to create logs from bulk upload
* create logs when processing bulk uploads
* remove bulk_upload_id from csv output
* create bulk upload logs only if all valid
- this will be changed later to allow for partial logs
- and only to create logs when a threshold has been met
* add method to blank invalid non setup fields
* bulk upload log creation blanks invalid fields
* fix incorrect logic for bulk upload renewal
* fix linting
* bulk upload log creation fail logs to sentry
* fix bulk upload line ending parsing
* extract bulk uploading csv parsing to class
* use csv parser in log creator
* change handle line endings mechanism
- we now strip all windows line endings for unix based line endings
- this normalises things making it simpler
* Enable multiple inferred check answers values
* lint and add new conditions to check answers for age1
* Rename method
* Refactor inferred answer value
* rebase update
* update inferred_check_answers_value after rebase
* Move the method to private
* Add a missing test
* rebase changes
* Add prefers not to say as the displayed value for all household members
* update [0] to .first
* replace [0] with .first
* able to view lettings bulk upload errors
* fix linting
* call service correctly in test
* add bulk upload sales questions mapping
* appease linter
* bulk upload error shows correct question
- depending on log type it will show relevant question for the field
concerned
* improve namespacing of classes
* add job to process bulk uploads
* move validation from parser to model
* add validations for field_1
* add validation for field_4
* pending tests for field_4
* convert field_mapping to array of hashes
* validate nulls based on form question
* actually load forms when toggling between forms
* validate null for startdate
* row parser has access to bulk upload
* csv upload validates first form section
* add postcode validation
* Refactor error mappings for row parser
* Add unittype question
* Fix null error setting and add builtype
* add wchair to bulk upload
* Add beds to bulk upload
* Add joint to bulk upload
* Add startertenancy to the bulk upload
* Add tenancy for bulk upload
* Add declaration to the bulk upload
* Add age1 and age1_known to bulk upload
* add ages to bulk upload
* add sex1 to bulk upload
* add ethnic_group and ethnic to bulk upload
* add national to bulk upload
* add ecstat1 to bulk upload
* add military related fields to bulk upload
* add preg_occ to bulk upload
* add housingneeds to bulk upload
* add illness to bulk upload
* add layear to bulk upload
* add waityear to bulk upload
* add reason to bulk upload
* add prevten to bulk upload
* add homeless to bulk upload
* add previous postcode to bulk upload
* add reasonable preferences to bulk upload
* add allocations system to bulk upload
* add referral to bulk upload
* add net_income_known to bulk upload
* add hb to bulk upload
* add benefits to bulk upload
* add rent fields to bulk upload
* add hhmemb to bulk upload
* use 2022 csv fixtures for bulk upload
* fix renewal mapping for bulk upload
* placeholder test for bulk upload validation
* fix bulk upload mapping for homeless field
* fix leftreg mapping for bulk upload
* fix user associations in bulk upload tests
* add gender fields for bulk upload
* add ecstatN fields to bulk upload
* add #relatN fields to bulk upload
* extract old_visible_id in factory to trait
* map net_income_known correctly for bulk upload
* fix income bugs for bulk upload
* add unitletas to bulk upload
* add #rsnvac to bulk upload
* add #sheltered to bulk upload
* add illness fields to bulk upload
* add #irproduct_other to bulk upload
* infer renewal from rsnvac for bulk upload
* add #tenancyother to bulk upload
* add #tenancylength to bulk upload
* bulk upload earnings accepts pennies but rounds
* add #reasonother to bulk upload
* fix mapping of #ppcodenk for bulk upload
* add #household_charge to bulk upload
* add #chcharge to bulk upload
* add #tcharge to bulk upload
* add #supcharg to bulk upload
* add pscharge to bulk upload
* add #scharge to bulk upload
* use case statement for bulk upload allocation
* add offered to bulk upload
* add propcode to bulk upload
* add major repair fields to bulk upload
* add #voiddate to bulk upload
* support YY year format for bulk upload
* test postcode strips whitespace for bulk upload
* add #la to bulk upload
* add previous la to bulk upload
* fix failing test
* remove duplicate line from rebase
* add first time social housing to bulk upload
* make methods private
* fix field_4 validation for bulk upload
- the null check was inverted by mistake
Co-authored-by: Kat <katrina@kosiak.co.uk>
Arthur Campbell
kosiakkatrina
Rachael Booth
natdeanlewissoftwire
Jack
Phil Lee
James Rose