* Check if response is http success
* Fix code scanning alert no. 65: Incomplete regular expression for hostnames
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
---------
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
* Make full stops consistent in error messages
* Fix test
* Update few hardcoded errors
* Update join in remaining row parsers
* Update some more tests
* Update more hard coded BU errors
* Fix typo I introduced - bedrooms to bedroom
---------
Co-authored-by: Kat <katrina@kosiak.co.uk>
Co-authored-by: kosiakkatrina <54268893+kosiakkatrina@users.noreply.github.com>
* update factory completed traits using an after build hook so that trait does what it says regardless of start or sale date of log
* use completed trait rather than completed2024 trait from factories
skip validation on saving logs from previous years so that logs created too far in the past do not trigger validations
* remove sale and start dates from logs in tests where that should not be important
build rather than create to reduce db interactions
in order that the setup is complete, build the necessary user and organisation with ids
* fix a wide range of tests affected by the changes in log factories completed traits
* write helper method to support having the correct rent period checkboxes checked
* update new and create in organisations controller and view to enable creation of relevant organisation rent periods
write tests for this
* small changes to models
* enable editing rent period in the UI
display change button on org details page
display rent periods question on edit page
* enable updating org rent periods
update logic in #update
related tests
* alter redirect after support user creates an organisation
* adjust various UI elements: ordering of rows, copy changes, label size and associated tests
* rework the #rent_period_labels method to return All under the correct conditions, rework tests related to that. + fix assorted tests that were either flakey or breaking due to addition of rent periods logic to create and update
* amend failing tests and resolve linting complaints
* changes following review
* disable checkboxes for rent periods if they are in use so that users are not able to make existing logs invalid
hint text added to the question to explain this
I have also added all rent periods to a hidden field to remove the need to fetch them again form the db in the update method
* update validation to reflect the fact that an org having no associated rent periods no longer means they accept all rent periods
update tests adding both cases and removing unnecessary additional db additions
* rake task to create rent period associations for orgs that have none
* revert mistaken copy changes in designs
* create rent periods in factories as default, with an option to skip. skip automatic creation in tests specifically related to rent periods
* stub api call for factory value, update csv tests and fixtures accordingly
* extract a good chunk of tests out of lettings_log_spec and into a dedicated derived fields spec file. in many cases refactor tests
* remove before(:context) and associated patterns.
use assign_attributes in various places for cleaner code
* escape . in regex for API call stubs to satisfy codeQL
remove destroy_all call at the start of a test that was dealing with leftover modesl in the test db
* further refactoring of various tests to reduce database interactions and improve speed
* remove outdated distinction between unitletas mappings from before 23/24
* remove tests that seem to be testing active record and/or ruby Date class
* write helper method to support having the correct rent period checkboxes checked
* update new and create in organisations controller and view to enable creation of relevant organisation rent periods
write tests for this
* small changes to models
* enable editing rent period in the UI
display change button on org details page
display rent periods question on edit page
* enable updating org rent periods
update logic in #update
related tests
* alter redirect after support user creates an organisation
* adjust various UI elements: ordering of rows, copy changes, label size and associated tests
* rework the #rent_period_labels method to return All under the correct conditions, rework tests related to that. + fix assorted tests that were either flakey or breaking due to addition of rent periods logic to create and update
* amend failing tests and resolve linting complaints
* changes following review
* disable checkboxes for rent periods if they are in use so that users are not able to make existing logs invalid
hint text added to the question to explain this
I have also added all rent periods to a hidden field to remove the need to fetch them again form the db in the update method
* update validation to reflect the fact that an org having no associated rent periods no longer means they accept all rent periods
update tests adding both cases and removing unnecessary additional db additions
* rake task to create rent period associations for orgs that have none
* revert mistaken copy changes in designs
* create rent periods in factories as default, with an option to skip. skip automatic creation in tests specifically related to rent periods
* stub api call for factory value, update csv tests and fixtures accordingly
* extract a good chunk of tests out of lettings_log_spec and into a dedicated derived fields spec file. in many cases refactor tests
* remove before(:context) and associated patterns.
use assign_attributes in various places for cleaner code
* escape . in regex for API call stubs to satisfy codeQL
remove destroy_all call at the start of a test that was dealing with leftover modesl in the test db
* further refactoring of various tests to reduce database interactions and improve speed
* remove outdated distinction between unitletas mappings from before 23/24
* remove tests that seem to be testing active record and/or ruby Date class
* Filter out empty options from selected filters
* Redirect to year filter if it is not selected
* Add remaining filter questions
* Display lettings filters CYA
* Add cancel, conditionally display send email
* Refactor
* Add removed tests
* Update method name
* Add sales logs filters
* Update tests
* Set label size on filter questions
* Refactor
* Deduplicate filter views
* Rename methods
* Authenticate pages
* Update filtering per organisation
* Update routing from CYA
* lint
* Refactor check_your_answers_filters_list
* Redirect if year is not given for sales csv
* Update formatted organisations methods
* Update missing and wrong copy
* Update cancel and back buttons
* CLDC-3442 Download logs csv per year (#2404)
* Specify year in lettings csv download service
* Specify year in sales csv download service
* Define non_question_fields per year
* Tidy up missing fields and tests
* Fix request tests
* Refactor
* Add missing param
* CLDC-3348 Update sales CSV for support (#2407)
* Update sales csv for support
* CLDC-3352 Update sales CSV address order (#2410)
* Update sales support csv address ordering
* Refactor
* CLDC-3367 Rename sales support CSV fields (#2413)
* Rename sales support CSV fields
* Update sales CSV user fields order (#2416)
* CLDC-2586 Rename scheme sensitive (#2405)
* Rename scheme sensitive in lettings csv
* Rename scheme sensitive in schemes csv
* CLDC-3347 Show soft validations in support CSVs only (#2406)
* Show soft validations in support csvs only
* CLDC-3352 Update lettings CSV address order (#2409)
* Update lettings support csv address ordering
* Refactor
* CLDC-3415/3421/3422 Add lettings CSV updates (#2415)
* Add first_time_property_let_as_social_housing to coordinators csv
* Update lettings CSV user fields order
* Remove new_old from lettings CSV
* Remove duplicate rent value check from CSV (#2423)
* Refactor CSV service (#2425)
* Extract and move lettings methods
* Extract and more sales methods
* Empty
* CLDC-3345 Rename created_by to assigned_to (#2372)
* Rename created_by to assigned_to
* Replace created_by with assigned_to
* Update created_by to assigned_to in exports, remove blank assigned to
* CLDC-3345 Add and set created_by fields (#2373)
* Add created_by
* Update existing created_by values
* Set created_by on single log
* Set created_by on BU
* Add created_by to exports
* feat: update since last merge
---------
Co-authored-by: natdeanlewissoftwire <nat.dean-lewis@softwire.com>
* Fix papertrail create version
---------
Co-authored-by: natdeanlewissoftwire <nat.dean-lewis@softwire.com>
* Update feature tests
* Remove old rake tasks
* Update some unit tests
* Update more tests
* More tests
* Fix failing sales household validation test
---------
Co-authored-by: Robert Sullivan <robert.sullivan@softwire.com>
* Pluralise buyer in household characteristics/setup
* Pluralise buyer in sales information sections
* Pluralise buyer in savings
* Pluralise other buyers, update privacy notice test
* Pluralise buyer in validations and translations
* Route to pages when joint purchase is not answered
* Fix apostrophes
* Update translations
* More updates
* DIsplay missing stock owners banner
* Extract the helpdesk url constant
* Add test, fix copy order
* Display banner to support
* Hide duplicate logs banner if other banners are present
* feat: add schemes and locations download links and pages
* feat: update current path helper
* feat: update tests for different user visibility levels
* feat: update search caption tests
* refactor: lint tests
* refactor: lint tests
* git: revert unintentional inclusion
* feat: update tests
* refactor: lint
* feat: DRY up routing
* refactor: lint
* feat: add csv confirmation view
* feat: add scheme csv service
* feat: rename
* feat: update csv service
* feat: update csv service
* feat: update controller and rename view
* feat: update view
* refactor: lint
* feat: show correct headers in csv
* feat: add locations and combined csv behaviour
* feat: remove redundant user instance variable
* feat: add scheme csv service spec
* feat: add scheme email csv job tests
* feat: update filters in spec
* refactor: move scheme_email_csv_job_spec.rb
* feat: update spec
* refactor: remove blank line
* feat: add nowrap to all download links
* feat: update org schemes controller with org schemes (and rename for clarity)
* feat: update link indentation and spec
* feat: only include location LA name, and rename to location_local_authority
* feat: update seed locations with westminster local authorities to avoid similar confusion to some that arose in PO review
* feat: display multiple active periods on a single line
* feat: display multiple active periods on a single line
* feat: update line spacing in search captions
* feat: replace 2/3 with full column in download page
* feat: move scheme alphabeticising into manager
* feat: update tests now search/filterless copy has changed
* refactor: lint
* refactor: lint
* refactor: lint
* feat: add filter alphabeticising test
* feat: correct spacing
* WIP commit
* feat:tidy up routing and fix scheme tab current bug
* feat: revert user_cell update
* feat: use latest merge date/start date where available for availability text for schemes and locations
* feat: add tests for availability text
* feat: include schema
* feat: use startdate instead to track merges on schemes and locations
* feat: test startdates are set correctly on merged schemes and locations
* feat: update schema
* refactor: lint
* feat: add migration file
* feat: update schema
* feat: add back links
* feat: tidy navigation bar behaviour
* feat: update scope auth
* faet: update page titles
* feat: simplify titles and revert some bulk upload dependent navigation behaviour
* feat: update support user tab titles and search bar text
* feat: give support users breadcrumbs to individual org obejcts for schemes, users and logs
* feat: add user param to log breadcrumbs
* refactor: erblint
* feat: update schema
* Revert "Merge branch 'CLDC-2996-merged-scheme-activation-date-bug' into CLDC-2691-support-nav-bug"
This reverts commit 37d4b1e194, reversing
changes made to 1d76829922.
* feat: update tests
* feat: update tests
* feat: simplify log type methods
* feat: simplify log title
* feat: update locations tab breadcrumbs to show owning scheme
* feat: use correct scheme path
* feat: update breadcrumbs in check ansewrs and review pages
* feat: update tests
* feat: only show editable duplicate sets for user and org
* feat: simplify and refactor
* feat: update tests
* feat: add and update tests
* feat: reset not routed questions twice to catch newly not routed questions after the first set are cleared
* Revert "feat: reset not routed questions twice to catch newly not routed questions after the first set are cleared"
This reverts commit 3007284670.
* TEST FOR PO TO BE REVERTED
* TEST FOR PO TO BE REVERTED (set back to dec 31)
* TEST UPDATE TESTS
* REVERT THIS TEST UPDATE
* REVERT THIS TEST UPDATE
* feat: timecop tests
* refactor: lint
* feat: add update methods
* feat: update tests (and TO BE REVERTED: change collection year)
* feat: update tests
* feat: update tests
* feat: update form end_date tests REVERT LATER
* feat: update tests
* refactor: lint
* feat: revert to previous end dates now passed PO review
* feat: use scoping to simplify editable set logic
* Move clear search, pass search when filtering
* Update search/filter caption content
* Put download links on the next line
* set filters_count to 0 where there are no filters
* Correctly display delete logs for support
* Update tests
* persist search after clearing filters
* create helper module to find all logs for a given user that have duplicates
write associated tests
update methods on user model to retrieve logs related to a given user
create a scope on logs to retrieve logs created by a given user
* check for duplicates and display notification banner on the index page where appropriate
tests to cover the desired behaviour
* alter duplicate logs helper to return duplicates in a format that can be passed through the params
update associated tests
* create user journey to view duplicate sets adn navigate to a given set
create a new route and add it to the link from the logs index page
create controller method and related view file
* update path for duplicate logs in routes, update factories, refactor duplicates scope to eliminate raw sql
* write tests for the duplicate logs index page
* correct linting errors
* minor amendments from code review.
* temp
* Refactor
* Create a scope for duplicate groups
* fixes
* lint
* Update controller specs
* Put duplicate sets count behind a feature flag
* refactor index
* Update duplicate_sets_count
* Update scopes
* tests
* pass in original log id
* tests
* Add duplicate index page content
---------
Co-authored-by: Kat <katrina@kosiak.co.uk>
* Save total logs count for bulk upload
* Update content of fix choice and confirm pages
* update bulk upload banner
* Rebase changes
* Extract fix choice summary into method
* Add comma
* feat: wip commit
* feat: rest of assigned to filter behaviour
* feat: start to update tests
* feat: update tests
* feat: update tests
* refactor: linting
* feat: add new test
* feat: add new test
* feat: update filters_count
* feat: let support users see all users
* feat: let users select users from stock owners/managing agents too
* refactor: typo
* feat: update tests
* CLDC-2494: WIP
* CLDC-2494: wip
* CLDC-2494: page work in progress
* cleanup
* Add a path for duplicate logs
* Display all duplicate logs
* Move a test
* Display duplicate check answers for logs
* Add buttons to delete duplicates
* Add a route for sales logs duplicates
* Update duplicated page to work for sales logs
* lint
* Add auth
* Rebase updates
* Update displayed questions
* Add delete duplicates page
* Update page content for when there are multiple duplicates
* Add auth to the path
* Update delete button to use delete controller
* Update page for sales logs
* Update styling
* Update success banner for deleting duplicates
* Typo
* Render not found if there are no duplicate logs
* rebase changes
* Rebase changes
* Update back and cancel links
* Update the duplicate journey after deleting logs
* Update change button routing
* Refactor tests
* Add content for no duplicates
* Update params
* Refactor tests and paths
* Update params to include original_log_id from the beginning
* Rename file
* lint
* Add full stop after deletion messages
* Add caption and update duplicate log IDs in the banner
* lint
* Styling
---------
Co-authored-by: Aaron Spencer <aaron.spencer@softwire.com>
* Redirect lettings log to review page if the collection year is closed for editing
* Redirect sales log to review page if the collection year is closed for editing
* Update location default deactivation date
* Update location default deactivation date
* Do not display resources for a closed collection year
* lint
* Extract a method
* tests
* Update breadcrumbs for closed period
* Renme end_date to new_logs_end_date
* Display change buttons in CYA if the collection is still open for editing
* Allow navigating to question if the collection is still open for editing
* Allow logs to be edited if the date is before edit end date
* Update sales validation to allow editing existing logs
* update tests
* Update edit_end_date
* Update some test wording
* Update new logs end date
* tests
* add a button to the logs list to delete multiple logs
style and position of button
helpers for displaying the button conditionally depending on user role and what filters and search are active
* correct indentation from 4 spaces to 2 in view file
* test appearance of delete logs button on index page for lettings logs
* write a happy path feature test for the entire journey
* create basic tests for the view component for listing logs to delete
* create request tests for the GET delete-logs path
* create request tests for the GET delete-logs-confirmation path
* create request tests for the DELETE delete-logs path
* comprehensive reworking after code review
ensure that we are not passing lists of ids through params in the query string, risking overflowing the maximum URL length,
adjust tests accordingly,
do not attempt to reuse the same table for sales and lettings
* alter config to allow creating controllers from the command line with associated spec files that matches how we test
* extract controller methods and associated tests to do with the delete logs feature into their own controller,
amend routes accordingly
* implement same work for sales as for lettings
* implement the story for lettings and sales logs under the organisation tab
routing and controller methods
testing for deleting sales logs, lettings or sales logs for an organisation
move storage of relevant routes inside the form object as a comprehensive view model
* merge the delete pages for lettings logs and sales logs, add to the tests for the lettings page to test sales specific content
* minor refactor to delete logs controller: ensure session filters are only fetched from teh session when needed and extract discard logs method to private method
* extract tables for lettings and sales to own partials
* refactor delete logs controller after tech review
improve the private method that builds the form object so that it has the flexibility to do so for all controller methods
ensure that the search term is passed to the delete logs controller when navigating through the organisations tab
ensure that noly logs for that organisation are displayed when navigating to delete logs through the organisations tab
* remove unnecessary untested arguments
* test new helper methods
* implement dirty fiddle to get the checkboxes smaller and also not misaligned
* ensure delete logs button is always visible on log lists when in the organisations tab
* minor linting corrections
* revert change, causing errors and outside the scope of this ticket
* simplify tests for whether delete logs button appears on index page
* replicate request specs from lettings for sales and organisations controllers
* minor refactor of lettings log feature spec setup, replicate happy path for sales
* minor refactors after rebasing onto Nat's work
* temp
* write tests for the delete logs form object
* lint: add new line at end of file
* respond to PO feedback
the log id in the delte logs table should be a link to the log
the delete logs button should be visible when the user is in a bulk upload journey
updated associated tests
# Context
- https://digital.dclg.gov.uk/jira/browse/CLDC-2316
- Implement bulk upload sales for new collection year 2023
- This is a first pass implementation and will probably have some bugs in it and we can address over time
# Changes
- Add CSV parser for sales 2023 to handle CSV structure
- Tweak collection window validation so error now contextual to year selected for upload
- Handle arbitrary ordering of CSV columns
- Fix ordering of errors in report and now ordered by cell
- Added `Upload your file again` link styled as button on error report to match lettings experience
- Update tooling to convert logs to 2023 csv rows with support for random column ordering
# Known issues
- There seem to be some issues with how UPRN is handled if the UPRN cannot be validated.
- For the above I think there is dependency on https://github.com/communitiesuk/submit-social-housing-lettings-and-sales-data/pull/1570 which should clear any errored fields so users can continue to create logs and fix within the service
* rename variables in test file
* write tests for new validations
* validate that answers to numeric questions are given to the correct level of accuracy and in normal format with only digits
* add error messages for new validations
* add or alter step on numeric questions in sales, amend one test given step changes
* copy change
* remove validation that is specific to number format and fold that functionality into accuracy validation, rejig tests accordingly
* fix various tests
* remove reference to currency in validation message that concerns step only and doesn't check whether firled is currency related
* alter copy to avoid redundant repetition in validaiton message
* save changes made in rebase
* changes after PO review
two step values had been missed in previous work
in various places there were custom validations applied that duplicated the functionality of the generic min max validation that is applied to all numeric questions, in these situations sometimes the min and max on the question class was inconsistent with the limit that triggered validations
these have been corrected and made consistent
various tests were affected by this and have been amended accordingly
* remove tests introduced by mistake in rebase
* amend step value on earnings
* rename variables in test file
* write tests for new validations
* validate that answers to numeric questions are given to the correct level of accuracy and in normal format with only digits
* add error messages for new validations
* add or alter step on numeric questions in sales, amend one test given step changes
* copy change
* remove validation that is specific to number format and fold that functionality into accuracy validation, rejig tests accordingly
* fix various tests
* remove reference to currency in validation message that concerns step only and doesn't check whether firled is currency related
* alter copy to avoid redundant repetition in validaiton message
* save changes made in rebase
* changes after PO review
two step values had been missed in previous work
in various places there were custom validations applied that duplicated the functionality of the generic min max validation that is applied to all numeric questions, in these situations sometimes the min and max on the question class was inconsistent with the limit that triggered validations
these have been corrected and made consistent
various tests were affected by this and have been amended accordingly
* remove tests introduced by mistake in rebase
* add first page for bulk upload resume journey
* bulk upload resume handles upload again
* add confirm page to bulk upload resume journey
* replace placeholder count with correct value
* apply recommendation for bulk upload resume choice
* add how to fix bulk upload mailer
* integrate new bulk upload approve journey
* add missing bulk upload error mappings
* remove test
* prevent approve being called multiple times
* bulk upload creates invisible logs ahead of time
* work invisible logs into bulk upload flow
* sort errors so deterministic
* remove unused ensure
* remove expected_log_count and processed
- these fields are no longer used or needed
* introduce pending status
* swap visible for pending logs
* only show visible lettings logs
* hard code status filters
* remove unused model methods
* only show visible sales logs
* form controller ignores hidden logs
* locations and schemes only affect visible logs
* Fix possible bulk upload years
* Initialise 2021/22 form routes for tests
* Return test routes from form handler
* Fix some tests
* fix bulk upload tests
- change from mocks to actual code calls
* update years for bulk upload form
* Fix some uprn tests
* Set the scheme as available from previous year if the collection is in crossover
* Update location available_from
* Update location and scheme tests
* Refactor and comment out a scheme test
* Some uprn and export tests
* Fix lettings logs controller tests and comment out previous collection message test
* fix tests for new collection year
* fix bulk upload tests
for some reason tenancylength must not be given for validation to pass
* update bulk upload tests for new year
* More fixes
* Undo location changes
* Fix remaining uprn
* further new collection year test fixes
---------
Co-authored-by: Phil Lee <asmega@users.noreply.github.com>
* add a validation against the case where monthly leashold charges are zero, test this, remove unnecessary validation on cash discount as this is covered by validate_numeric_min_max
* remove focus: true from describe block
* minor changes for testing and linting
* correct minor bug found in PO
* make minor amendments to failing tests
Manny Dinssa
kosiakkatrina
Arthur Campbell
natdeanlewissoftwire
Jack
Phil Lee