baarkerlounger
/
submit-social-housing-lettings-and-sales-data
mirror of https://github.com/communitiesuk/submit-social-housing-lettings-and-sales-data.git
1
0
Fork 0
Code Issues Projects Releases Wiki Activity
Browse Source

CLDC-1779 Bulk upload lettings validation (#1148) 

* able to view lettings bulk upload errors

* fix linting

* call service correctly in test

* add bulk upload sales questions mapping

* appease linter

* bulk upload error shows correct question

- depending on log type it will show relevant question for the field
  concerned

* improve namespacing of classes

* add job to process bulk uploads

* move validation from parser to model

* add validations for field_1

* add validation for field_4

* pending tests for field_4

* convert field_mapping to array of hashes

* validate nulls based on form question

* actually load forms when toggling between forms

* validate null for startdate

* row parser has access to bulk upload

* csv upload validates first form section

* add postcode validation

* Refactor error mappings for row parser

* Add unittype question

* Fix null error setting and add builtype

* add wchair to bulk upload

* Add beds to bulk upload

* Add joint to bulk upload

* Add startertenancy to the bulk upload

* Add tenancy for bulk upload

* Add declaration to the bulk upload

* Add age1 and age1_known to bulk upload

* add ages to bulk upload

* add sex1 to bulk upload

* add ethnic_group and ethnic to bulk upload

* add national to bulk upload

* add ecstat1 to bulk upload

* add military related fields to bulk upload

* add preg_occ to bulk upload

* add housingneeds to bulk upload

* add illness to bulk upload

* add layear to bulk upload

* add waityear to bulk upload

* add reason to bulk upload

* add prevten to bulk upload

* add homeless to bulk upload

* add previous postcode to bulk upload

* add reasonable preferences to bulk upload

* add allocations system to bulk upload

* add referral to bulk upload

* add net_income_known to bulk upload

* add hb to bulk upload

* add benefits to bulk upload

* add rent fields to bulk upload

* add hhmemb to bulk upload

* use 2022 csv fixtures for bulk upload

* fix renewal mapping for bulk upload

* placeholder test for bulk upload validation

* fix bulk upload mapping for homeless field

* fix leftreg mapping for bulk upload

* fix user associations in bulk upload tests

* add gender fields for bulk upload

* add ecstatN fields to bulk upload

* add #relatN fields to bulk upload

* extract old_visible_id in factory to trait

* map net_income_known correctly for bulk upload

* fix income bugs for bulk upload

* add unitletas to bulk upload

* add #rsnvac to bulk upload

* add #sheltered to bulk upload

* add illness fields to bulk upload

* add #irproduct_other to bulk upload

* infer renewal from rsnvac for bulk upload

* add #tenancyother to bulk upload

* add #tenancylength to bulk upload

* bulk upload earnings accepts pennies but rounds

* add #reasonother to bulk upload

* fix mapping of #ppcodenk for bulk upload

* add #household_charge to bulk upload

* add #chcharge to bulk upload

* add #tcharge to bulk upload

* add #supcharg to bulk upload

* add pscharge to bulk upload

* add #scharge to bulk upload

* use case statement for bulk upload allocation

* add offered to bulk upload

* add propcode to bulk upload

* add major repair fields to bulk upload

* add #voiddate to bulk upload

* support YY year format for bulk upload

* test postcode strips whitespace for bulk upload

* add #la to bulk upload

* add previous la to bulk upload

* fix failing test

* remove duplicate line from rebase

* add first time social housing to bulk upload

* make methods private

* fix field_4 validation for bulk upload

- the null check was inverted by mistake

Co-authored-by: Kat <katrina@kosiak.co.uk>
CLDC-846-demographic-gender-validation
Phil Lee 2 years ago committed by GitHub
parent
6a7795267f
commit
90480d1af5
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
10 changed files with 1411 additions and 61 deletions
Whitespace
Show all changes Ignore whitespace when comparing lines Ignore changes in amount of whitespace Ignore changes in whitespace at EOL
Split View
Diff Options
Show Stats Download Patch File Download Diff File
  1. 12
      app/models/form_handler.rb
  2. 1
      app/models/lettings_log.rb
  3. 644
      app/services/bulk_upload/lettings/row_parser.rb
  4. 16
      app/services/bulk_upload/lettings/validator.rb
  5. 1
      spec/factories/bulk_upload.rb
  6. 4
      spec/factories/organisation.rb
  7. 72
      spec/fixtures/files/2022_23_lettings_bulk_upload.csv
  8. 716
      spec/services/bulk_upload/lettings/row_parser_spec.rb
  9. 2
      spec/services/bulk_upload/lettings/validator_spec.rb
  10. 4
      spec/services/bulk_upload/processor_spec.rb

12
app/models/form_handler.rb
Unescape View File

@ -78,6 +78,16 @@ class FormHandler
forms.count { |form| form.start_date < now && now < form.end_date } > 1
end
def use_fake_forms!
@directories = ["spec/fixtures/forms"]
@forms = get_all_forms
end
def use_real_forms!
@directories = ["config/forms"]
@forms = get_all_forms
end
private
def get_all_forms
@ -85,6 +95,6 @@ private
end
def directories
Rails.env.test? ? ["spec/fixtures/forms"] : ["config/forms"]
@directories ||= Rails.env.test? ? ["spec/fixtures/forms"] : ["config/forms"]
end
end

1
app/models/lettings_log.rb
Unescape View File

@ -8,6 +8,7 @@ class LettingsLogValidator < ActiveModel::Validator
include Validations::TenancyValidations
include Validations::DateValidations
include Validations::LocalAuthorityValidations
def validate(record)
validation_methods = public_methods.select { |method| method.starts_with?("validate_") }
validation_methods.each { |meth| public_send(meth, record) }

644
app/services/bulk_upload/lettings/row_parser.rb
Unescape View File

@ -2,6 +2,8 @@ class BulkUpload::Lettings::RowParser
include ActiveModel::Model
include ActiveModel::Attributes
attribute :bulk_upload
attribute :field_1, :integer
attribute :field_2
attribute :field_3
@ -51,7 +53,7 @@ class BulkUpload::Lettings::RowParser
attribute :field_47, :integer
attribute :field_48, :integer
attribute :field_49, :integer
attribute :field_50, :integer
attribute :field_50, :decimal
attribute :field_51, :integer
attribute :field_52, :integer
attribute :field_53, :string
@ -137,49 +139,647 @@ class BulkUpload::Lettings::RowParser
attribute :field_133, :integer
attribute :field_134, :integer
validates :field_1, presence: true, numericality: { in: (1..12) }
validates :field_4, numericality: { in: (1..999), allow_blank: true }
validates :field_4, presence: true, if: :field_4_presence_check
validates :field_1, presence: true, inclusion: { in: (1..12).to_a }
validates :field_4, presence: { if: proc { [2, 4, 6, 8, 10, 12].include?(field_1) } }
validates :field_96, presence: true
validates :field_97, presence: true
validates :field_98, presence: true
def valid?
errors.clear
super
validate_data_types
validate_nulls
validate :validate_possible_answers
log.valid?
# delegate :valid?, to: :native_object
# delegate :errors, to: :native_object
log.errors.each do |error|
fields = field_mapping_for_errors[error.attribute] || []
fields.each { |field| errors.add(field, error.type) }
end
errors.blank?
end
def log
@log ||= LettingsLog.new(attributes_for_log)
end
private
def native_object
@native_object ||= LettingsLog.new(attributes_for_log)
def attribute_set
@attribute_set ||= instance_variable_get(:@attributes)
end
def validate_data_types
unless attribute_set["field_1"].value_before_type_cast&.match?(/\A\d+\z/)
errors.add(:field_1, :invalid)
end
end
def postcode_full
"#{field_108} #{field_109}" if field_108 && field_109
end
def postcode_known
if postcode_full.present?
1
elsif field_107.present?
0
end
end
def questions
log.form.subsections.flat_map { |ss| ss.applicable_questions(log) }
end
def field_mapping
def validate_nulls
field_mapping_for_errors.each do |error_key, fields|
question_id = error_key.to_s
question = questions.find { |q| q.id == question_id }
next unless question
next if log.optional_fields.include?(question.id)
next if question.completed?(log)
fields.each { |field| errors.add(field, :blank) }
end
end
def field_mapping_for_errors
{
field_134: :renewal,
lettype: [:field_1],
tenancycode: [:field_7],
postcode_known: %i[field_107 field_108 field_109],
postcode_full: %i[field_107 field_108 field_109],
la: %i[field_107],
owning_organisation_id: [:field_111],
managing_organisation_id: [:field_113],
renewal: [:field_134],
scheme: %i[field_4 field_5],
created_by: [],
needstype: [],
rent_type: %i[field_1 field_129 field_130],
startdate: %i[field_98 field_97 field_96],
unittype_gn: %i[field_102],
builtype: %i[field_103],
wchair: %i[field_104],
beds: %i[field_101],
joint: %i[field_133],
startertenancy: %i[field_8],
tenancy: %i[field_9],
tenancyother: %i[field_10],
tenancylength: %i[field_11],
declaration: %i[field_132],
age1_known: %i[field_12],
age1: %i[field_12],
age2_known: %i[field_13],
age2: %i[field_13],
age3_known: %i[field_14],
age3: %i[field_14],
age4_known: %i[field_15],
age4: %i[field_15],
age5_known: %i[field_16],
age5: %i[field_16],
age6_known: %i[field_17],
age6: %i[field_17],
age7_known: %i[field_18],
age7: %i[field_18],
age8_known: %i[field_19],
age8: %i[field_19],
sex1: %i[field_20],
sex2: %i[field_21],
sex3: %i[field_22],
sex4: %i[field_23],
sex5: %i[field_24],
sex6: %i[field_25],
sex7: %i[field_26],
sex8: %i[field_27],
ethnic_group: %i[field_43],
ethnic: %i[field_43],
national: %i[field_44],
relat2: %i[field_28],
relat3: %i[field_29],
relat4: %i[field_30],
relat5: %i[field_31],
relat6: %i[field_32],
relat7: %i[field_33],
relat8: %i[field_34],
ecstat1: %i[field_35],
ecstat2: %i[field_36],
ecstat3: %i[field_37],
ecstat4: %i[field_38],
ecstat5: %i[field_39],
ecstat6: %i[field_40],
ecstat7: %i[field_41],
ecstat8: %i[field_42],
armedforces: %i[field_45],
leftreg: %i[field_114],
reservist: %i[field_46],
preg_occ: %i[field_47],
housingneeds: %i[field_47],
illness: %i[field_118],
layear: %i[field_66],
waityear: %i[field_67],
reason: %i[field_52],
reasonother: %i[field_53],
prevten: %i[field_61],
homeless: %i[field_68],
prevloc: %i[field_62],
previous_la_known: %i[field_62],
ppcodenk: %i[field_65],
ppostcode_full: %i[field_63 field_64],
reasonpref: %i[field_69],
rp_homeless: %i[field_70],
rp_insan_unsat: %i[field_71],
rp_medwel: %i[field_72],
rp_hardship: %i[field_73],
rp_dontknow: %i[field_74],
cbl: %i[field_75],
chr: %i[field_76],
cap: %i[field_77],
referral: %i[field_78],
net_income_known: %i[field_51],
earnings: %i[field_50],
incfreq: %i[field_116],
hb: %i[field_48],
benefits: %i[field_49],
period: %i[field_79],
brent: %i[field_80],
scharge: %i[field_81],
pscharge: %i[field_82],
supcharg: %i[field_83],
tcharge: %i[field_84],
chcharge: %i[field_85],
household_charge: %i[field_86],
hbrentshortfall: %i[field_87],
tshortfall: %i[field_88],
unitletas: %i[field_105],
rsnvac: %i[field_106],
sheltered: %i[field_117],
illness_type_1: %i[field_119],
illness_type_2: %i[field_120],
illness_type_3: %i[field_121],
illness_type_4: %i[field_122],
illness_type_5: %i[field_123],
illness_type_6: %i[field_124],
illness_type_7: %i[field_125],
illness_type_8: %i[field_126],
illness_type_9: %i[field_127],
illness_type_10: %i[field_128],
irproduct_other: %i[field_131],
offered: %i[field_99],
propcode: %i[field_100],
majorrepairs: %i[field_92 field_93 field_94],
mrcdate: %i[field_92 field_93 field_94],
voiddate: %i[field_89 field_90 field_91],
}
end
def validate_possible_answers
field_mapping.each do |field, attribute|
possible_answers = FormHandler.instance.current_lettings_form.questions.find { |q| q.id == attribute.to_s }.answer_options.keys
def startdate
Date.new(field_98 + 2000, field_97, field_96) if field_98.present? && field_97.present? && field_96.present?
end
def renttype
case field_1
when 1, 2, 3, 4
:social
when 5, 6, 7, 8
:affordable
when 9, 10, 11, 12
:intermediate
end
end
unless possible_answers.include?(public_send(field))
errors.add(field, "Value supplied is not one of the permitted values")
def rent_type
case renttype
when :social
Imports::LettingsLogsImportService::RENT_TYPE[:social_rent]
when :affordable
if field_129 == 1
Imports::LettingsLogsImportService::RENT_TYPE[:london_affordable_rent]
else
Imports::LettingsLogsImportService::RENT_TYPE[:affordable_rent]
end
when :intermediate
case field_130
when 1
Imports::LettingsLogsImportService::RENT_TYPE[:rent_to_buy]
when 2
Imports::LettingsLogsImportService::RENT_TYPE[:london_living_rent]
when 3
Imports::LettingsLogsImportService::RENT_TYPE[:other_intermediate_rent_product]
end
end
end
def owning_organisation_id
Organisation.find_by(old_visible_id: field_111)&.id
end
def managing_organisation_id
Organisation.find_by(old_visible_id: field_113)&.id
end
def attributes_for_log
hash = field_mapping.invert
attributes = {}
hash.map do |k, v|
attributes[k] = public_send(v)
end
attributes["lettype"] = field_1
attributes["tenancycode"] = field_7
attributes["la"] = field_107
attributes["postcode_known"] = postcode_known
attributes["postcode_full"] = postcode_full
attributes["owning_organisation_id"] = owning_organisation_id
attributes["managing_organisation_id"] = managing_organisation_id
attributes["renewal"] = renewal
attributes["scheme"] = scheme
attributes["created_by"] = bulk_upload.user
attributes["needstype"] = bulk_upload.needstype
attributes["rent_type"] = rent_type
attributes["startdate"] = startdate
attributes["unittype_gn"] = field_102
attributes["builtype"] = field_103
attributes["wchair"] = field_104
attributes["beds"] = field_101
attributes["joint"] = field_133
attributes["startertenancy"] = field_8
attributes["tenancy"] = field_9
attributes["tenancyother"] = field_10
attributes["tenancylength"] = field_11
attributes["declaration"] = field_132
attributes["age1_known"] = field_12.present? ? 0 : 1
attributes["age1"] = field_12
attributes["age2_known"] = field_13.present? ? 0 : 1
attributes["age2"] = field_13
attributes["age3_known"] = field_14.present? ? 0 : 1
attributes["age3"] = field_14
attributes["age4_known"] = field_15.present? ? 0 : 1
attributes["age4"] = field_15
attributes["age5_known"] = field_16.present? ? 0 : 1
attributes["age5"] = field_16
attributes["age6_known"] = field_17.present? ? 0 : 1
attributes["age6"] = field_17
attributes["age7_known"] = field_18.present? ? 0 : 1
attributes["age7"] = field_18
attributes["age8_known"] = field_19.present? ? 0 : 1
attributes["age8"] = field_19
attributes["sex1"] = field_20
attributes["sex2"] = field_21
attributes["sex3"] = field_22
attributes["sex4"] = field_23
attributes["sex5"] = field_24
attributes["sex6"] = field_25
attributes["sex7"] = field_26
attributes["sex8"] = field_27
attributes["ethnic_group"] = ethnic_group_from_ethnic
attributes["ethnic"] = field_43
attributes["national"] = field_44
attributes["relat2"] = field_28
attributes["relat3"] = field_29
attributes["relat4"] = field_30
attributes["relat5"] = field_31
attributes["relat6"] = field_32
attributes["relat7"] = field_33
attributes["relat8"] = field_34
attributes["ecstat1"] = field_35
attributes["ecstat2"] = field_36
attributes["ecstat3"] = field_37
attributes["ecstat4"] = field_38
attributes["ecstat5"] = field_39
attributes["ecstat6"] = field_40
attributes["ecstat7"] = field_41
attributes["ecstat8"] = field_42
attributes["details_known_2"] = details_known(2)
attributes["details_known_3"] = details_known(3)
attributes["details_known_4"] = details_known(4)
attributes["details_known_5"] = details_known(5)
attributes["details_known_6"] = details_known(6)
attributes["details_known_7"] = details_known(7)
attributes["details_known_8"] = details_known(8)
attributes["armedforces"] = field_45
attributes["leftreg"] = leftreg
attributes["reservist"] = field_46
attributes["preg_occ"] = field_47
attributes["housingneeds"] = housingneeds
attributes["illness"] = field_118
attributes["layear"] = field_66
attributes["waityear"] = field_67
attributes["reason"] = field_52
attributes["reasonother"] = field_53
attributes["prevten"] = field_61
attributes["homeless"] = homeless
attributes["prevloc"] = prevloc
attributes["previous_la_known"] = previous_la_known
attributes["ppcodenk"] = ppcodenk
attributes["ppostcode_full"] = ppostcode_full
attributes["reasonpref"] = field_69
attributes["rp_homeless"] = field_70
attributes["rp_insan_unsat"] = field_71
attributes["rp_medwel"] = field_72
attributes["rp_hardship"] = field_73
attributes["rp_dontknow"] = field_74
attributes["cbl"] = cbl
attributes["chr"] = chr
attributes["cap"] = cap
attributes["letting_allocation_unknown"] = letting_allocation_unknown
attributes["referral"] = field_78
attributes["net_income_known"] = net_income_known
attributes["earnings"] = earnings
attributes["incfreq"] = field_116
attributes["hb"] = field_48
attributes["benefits"] = field_49
attributes["period"] = field_79
attributes["brent"] = field_80
attributes["scharge"] = field_81
attributes["pscharge"] = field_82
attributes["supcharg"] = field_83
attributes["tcharge"] = field_84
attributes["chcharge"] = field_85
attributes["household_charge"] = field_86
attributes["hbrentshortfall"] = field_87
attributes["tshortfall_known"] = tshortfall_known
attributes["tshortfall"] = field_88
attributes["hhmemb"] = hhmemb
attributes["unitletas"] = field_105
attributes["rsnvac"] = rsnvac
attributes["sheltered"] = field_117
attributes["illness_type_1"] = field_119
attributes["illness_type_2"] = field_120
attributes["illness_type_3"] = field_121
attributes["illness_type_4"] = field_122
attributes["illness_type_5"] = field_123
attributes["illness_type_6"] = field_124
attributes["illness_type_7"] = field_125
attributes["illness_type_8"] = field_126
attributes["illness_type_9"] = field_127
attributes["illness_type_10"] = field_128
attributes["irproduct_other"] = field_131
attributes["offered"] = field_99
attributes["propcode"] = field_100
attributes["majorrepairs"] = majorrepairs
attributes["mrcdate"] = mrcdate
attributes["voiddate"] = voiddate
attributes["first_time_property_let_as_social_housing"] = first_time_property_let_as_social_housing
attributes
end
def field_4_presence_check
[1, 3, 5, 7, 9, 11].include?(field_1)
def first_time_property_let_as_social_housing
case rsnvac
when 15, 16, 17
1
end
end
def rsnvac
field_106
end
def voiddate
Date.new(field_91 + 2000, field_90, field_89) if field_91.present? && field_90.present? && field_89.present?
end
def majorrepairs
mrcdate.present? ? 1 : 0
end
def mrcdate
Date.new(field_94 + 2000, field_93, field_92) if field_94.present? && field_93.present? && field_92.present?
end
def prevloc
field_62
end
def previous_la_known
prevloc.present? ? 1 : 0
end
def ppcodenk
case field_65
when 1
1
when 2
0
end
end
def earnings
field_50.round if field_50.present?
end
def net_income_known
case field_51
when 1
0
when 2
1
when 3
1
when 4
2
end
end
def leftreg
case field_114
when 3
3
when 4
1
when 5
2
when 6
0
end
end
def homeless
case field_68
when 1
1
when 12
11
end
end
def renewal
case field_134
when 1
1
when 2
0
when nil
field_116 == 14 ? 1 : 0
end
end
def details_known(person_n)
send("person_#{person_n}_present?") ? 0 : 1
end
def hhmemb
[
person_2_present?,
person_3_present?,
person_4_present?,
person_5_present?,
person_6_present?,
person_7_present?,
person_8_present?,
].count(true) + 1
end
def person_2_present?
field_13.present? && field_21.present? && field_28.present?
end
def person_3_present?
field_14.present? && field_22.present? && field_29.present?
end
def person_4_present?
field_15.present? && field_23.present? && field_30.present?
end
def person_5_present?
field_16.present? && field_24.present? && field_31.present?
end
def person_6_present?
field_17.present? && field_25.present? && field_32.present?
end
def person_7_present?
field_18.present? && field_26.present? && field_33.present?
end
def person_8_present?
field_19.present? && field_27.present? && field_34.present?
end
def tshortfall_known
field_87 == 1 ? 0 : 1
end
def letting_allocation_unknown
[cbl, chr, cap].all?(0) ? 1 : 0
end
def cbl
case field_75
when 2
0
when 1
1
end
end
def chr
case field_76
when 2
0
when 1
1
end
end
def cap
case field_77
when 2
0
when 1
1
end
end
def ppostcode_full
"#{field_63} #{field_64}".strip.gsub(/\s+/, " ")
end
def housingneeds
if field_59 == 1
1
elsif field_60 == 1
3
else
2
end
end
def ethnic_group_from_ethnic
return nil if field_43.blank?
case field_43
when 1, 2, 3, 18
0
when 4, 5, 6, 7
1
when 8, 9, 10, 11, 15
2
when 12, 13, 14
3
when 16, 19
4
when 17
17
end
end
def scheme
@scheme ||= Scheme.find_by(old_visible_id: field_4)
end
end

16
app/services/bulk_upload/lettings/validator.rb
Unescape View File

@ -151,9 +151,6 @@ class BulkUpload::Lettings::Validator
end
def call
row_offset = 6
col_offset = 0
row_parsers.each_with_index do |row_parser, index|
row_parser.valid?
@ -166,7 +163,7 @@ class BulkUpload::Lettings::Validator
tenant_code: row_parser.field_7,
property_ref: row_parser.field_100,
row:,
cell: "#{cols[field_number_for_attribute(error.attribute) + col_offset]}#{row}",
cell: "#{cols[field_number_for_attribute(error.attribute) - col_offset + 1]}#{row + 1}",
)
end
end
@ -178,6 +175,14 @@ class BulkUpload::Lettings::Validator
private
def row_offset
5
end
def col_offset
1
end
def field_number_for_attribute(attribute)
attribute.to_s.split("_").last.to_i
end
@ -191,6 +196,7 @@ private
stripped_row = row[1..]
headers = ("field_1".."field_134").to_a
hash = Hash[headers.zip(stripped_row)]
hash[:bulk_upload] = bulk_upload
BulkUpload::Lettings::RowParser.new(hash)
end
@ -221,7 +227,7 @@ private
end
def body_rows
rows[6..]
rows[row_offset..]
end
def validate_file_not_empty

1
spec/factories/bulk_upload.rb
Unescape View File

@ -7,6 +7,7 @@ FactoryBot.define do
year { 2022 }
identifier { SecureRandom.uuid }
sequence(:filename) { |n| "bulk-upload-#{n}.csv" }
needstype { 1 }
trait(:sales) do
log_type { BulkUpload.log_types[:sales] }

4
spec/factories/organisation.rb
Unescape View File

@ -9,6 +9,10 @@ FactoryBot.define do
created_at { Time.zone.now }
updated_at { Time.zone.now }
holds_own_stock { true }
trait :with_old_visible_id do
old_visible_id { rand(9_999_999).to_s }
end
end
factory :organisation_rent_period do

72
spec/fixtures/files/2022_23_lettings_bulk_upload.csv vendored
Unescape View File

@ -0,0 +1,72 @@
Question,What is the letting type?,[BLANK],[BLANK],"Management group code
If supported housing","Scheme code
If supported housing",[BLANK],"What is the tenant code?
This is how you usually refer to this tenancy on your own systems",Is this a starter tenancy? ,What is the tenancy type? ,If 'Other' what is the tenancy type?,"What is the length of the fixed-term tenancy to the nearest year?
If fixed-term tenancy",Age of Person 1,Age of Person 2,Age of Person 3,Age of Person 4,Age of Person 5,Age of Person 6,Age of Person 7,Age of Person 8,Gender identity of Person 1,Gender identity of Person 2,Gender identity of Person 3,Gender identity of Person 4,Gender identity of Person 5,Gender identity of Person 6,Gender identity of Person 7,Gender identity of Person 8,Person 2's relationship to lead tenant,Person 3's relationship to lead tenant,Person 4's relationship to lead tenant,Person 5's relationship to lead tenant,Person 6's relationship to lead tenant,Person 7's relationship to lead tenant,Person 8's relationship to lead tenant,Working situation of Person 1,Working situation of Person 2,Working situation of Person 3,Working situation of Person 4,Working situation of Person 5,Working situation of Person 6,Working situation of Person 7,Working situation of Person 8,What is the lead tenant’s ethnic group? ,What is the lead tenant’s nationality? ,Does anybody in the household have links to the UK armed forces? ,Was the person seriously injured or ill as a result of serving in the UK armed forces? ,Is anybody in the household pregnant? ,Is the tenant likely to be receiving benefits related to housing? ,"How much of the household's income is from Universal Credit, state pensions or benefits? ","How much income does the household have in total?
Include any income after tax from employment, pensions, and Universal Credit
Don't include National Insurance (NI) contributions and tax, housing benefit, child benefit, or council tax support ",Do you know the household's income?,What is the tenant's main reason for the household leaving their last settled home? ,"If 'other', what was the main reason for leaving their last settled home?",[BLANK],"Disabled access needs
a) Fully wheelchair-accessible housing ","Disabled access needs
b) Wheelchair access to essential rooms ","Disabled access needs
c) Level access housing ","Disabled access needs
f) Other disabled access needs ","Disabled access needs
g) No disabled access needs ","Disabled access needs
h) Don’t know",Where was the household immediately before this letting? ,What is the local authority of the household's last settled home?,Part 1 of postcode of last settled home,Part 2 of postcode of last settled home ,Do you know the postcode of the last settled home?,How long has the household continuously lived in the local authority area of the new letting? ,How long has the household been on the local authority waiting list for the new letting? ,Was the tenant homeless directly before this tenancy?,Was the household given 'reasonable preference' by the local authority? ,"Reasonable Preference
They were homeless or about to lose their home (within 56 days) ","Reasonable Preference
They were living in unsanitary, overcrowded or unsatisfactory housing ","Reasonable Preference
They needed to move due to medical and welfare reasons (including disability) ","Reasonable Preference
They needed to move to avoid hardship to themselves or others ","Reasonable Preference
Don't know ",Was the letting made under Choice-Based Lettings (CBL)? ,Was the letting made under the Common Housing Register (CHR)? ,Was the letting made under the Common Allocation Policy (CAP)? ,What was the source of referral for this letting? ,How often does the household pay rent and other charges? ,What is the basic rent? ,What is the service charge? ,What is the personal service charge? ,What is the support charge? ,Total charge,"
If this is a care home,
how much does the household pay every [time period]?","Does the household pay rent or other charges for the accommodation?
If supported housing ","After the household has received any housing-related benefits, will they still need to pay basic rent and other charges? ",What do you expect the outstanding amount to be? ,What is the void or renewal day? - DD,What is the void or renewal month? - MM,What is the void or renewal year? - YYYY,What day were Major Repairs completed on? - DD,What month were Major Repairs completed on? - MM,What year were Major Repairs completed on? - YY,[BLANK],What day did the tenancy start? - DD,What month did the tenancy start? - MM,What year did the tenancy start? - YY,"Since becoming available, how many times has the property been previously offered? ","What is the property reference?
This is how you usually refer to this property on your own systems","How many bedrooms does the property have?
If general needs","What type of unit is the property?
If general needs","Which type of building is the property?
If general needs","Is the property built or adapted to wheelchair-user standards?
If general needs","What type was the property most recently let as?
If re-let",What is the reason for the property being vacant? ,"What is the local authority of the property?
If general needs","Part 1 of postcode of property
If general needs","Part 2 of postcode of property
If general needs",[BLANK],"Which organisation owns this property?
Organisation's CORE ID",Username Field,"Which organisation manages this property? 
Organisation's CORE ID",Is the person still serving in the UK armed forces? ,[BLANK],How often does the household receive income? ,"Is this letting in sheltered accommodation?
If supported housing",Does anybody in the household have a physical or mental health condition (or other illness) expected to last 12 months or more? ,"Vision, for example blindness or partial sight","Hearing, for example deatness or partial hearing","Mobility, for example walking short distances or climbing stairs","Dexterity, for example lifting and carrying objects, using a keyboard",Learning or understanding or concentrating,Memory,"Mental health, for example depression or anxiety",Stamina or breathing or fatigue,"Socially or behaviourally (e.g. associated with autism spectrum disorder (ASD) which includes Aspergers', or attention deficit hyperactivity disorder (ADHD))",Other illness or condition,Is this letting a London Affordable Rent letting?,Which type of Intermediate Rent is this letting?,Which 'Other' type of Intermediate Rent is this letting?,Has the tenant seen the DLUHC privacy notice? ,Is this a joint tenancy? ,Is this a renewal to the same tenant in the same property?,
Values,1 - 12,,,1 - 999,,,max 13 digits,1 - 2,2 - 7,Text ,1 - 99,15 - 120 or R,1 - 120 or R,,,,,,,"M, F, X or R",,,,,,,,"P,C,X or R",,,,,,,0 - 11,,,,,,,,1 - 19,"12 - 13, 17 - 19",1 - 6,1 - 3,,"1, 3, 6, 9",1 - 4,0 - 99999,1 - 4,"1 - 2, 4, 7 - 14, 16 - 20, 28 - 31 or 34 - 47",Text,,1 or null ,,,,,,"3- 4, 6 - 7, 9 - 10, 13 - 14, 18 - 19, 21 or 23 - 35",ONS CODE - E + 8 Digits,XXX(X),XXX,1 - 2,1 - 2 or 5 - 10,2 or 5 - 11,1 or 12,1 - 3,1 or null,,,,,1 or 2,,,"1 - 4, 7 - 10, 12 - 17",1 - 9,xxxx.xx,,,,,,1 or null,1 - 3,0 - 9999,1 - 31,1 - 12,19 - 23,1 - 31,1 - 12,13 - 23,,1 - 31,1 - 12,22-23,0+,12 Digits,1 - 7,"1, 2, 4 or 6 - 10",1 or 2,,1 - 4,"5, 6, 8 - 14, ",ONS CODE E + 9 digits,XXX(X),XXX,,Up to 7 digits,Username of CORE account this letting log should be assigned to,Up to 7 digits,3 - 6,,1 - 3,1 - 4,1 - 3,1 or null,,,,,,,,,,1 - 3,,Text,1,1 - 3,1 - 2,
Can be null?,No,,,"only if field 1 = 1, 3, 5, 7, 9 or 11",,,No,,,"Yes, if 9 is not 3","Yes, if 9 = 2, 3, 5 or 7",No,"Yes, if field 21, 28 and 36 are null","Yes, if 22, 29 and 37 are null","Yes, if 23, 30 and 38 are null","Yes, if 24, 31 and 39 are null","Yes, if 25, 32 and 40 are null","Yes, if 26, 33 and 41 are null","Yes, if 27, 34 and 42 are null", No,"Yes, if 13, 28 and 36 are null","Yes, if 14, 29 and 37 are null","Yes, if 15, 30 and 38 are null","Yes, if 16, 31 and 39 are null","Yes, if 17, 32 and 40 are null","Yes, if 18, 33 and 41 are null","Yes, if 19, 34 and 42 are null","Yes, if 13, 21 and 36 are null","Yes, if 14, 22 and 37 are null","Yes, if 15, 23 and 38 are null","Yes, if 16, 24 and 39 are null","Yes, if 17, 25 and 40 are null","Yes, if 18, 26 and 41 are null","Yes, if 19, 27 and 42 are null",No,"Yes, if 13, 21 and 28 are null","Yes, if 14, 22 and 29 are null","Yes, if 15, 23 and 30 are null","Yes, if 16, 24 and 31 are null","Yes, if 17, 25 and 32 are null","Yes, if 18, 26 and 33 are null","Yes, if 19, 27 and 34 are null",No,,,"Yes, must be null if 45 = 2 or 3;
no, if 45 = 1, 4 or 5",No,,,If 51 = 4,No,No,"Yes, if 52 is not 20",,"Selections are ((A or B or C)) - ((A, or B or C and F)) - ((F)) - ((G)) - ((H))",,,,,,No,,"Yes, if 65 = 1",,"Yes, if 63 and 64 contain full and valid entries",No,,,," If 69 = 1, select at least one of the 5 categories;
If 69 = 2 or 3, then Null.",,,,,No,,,,,Only if 85 or 86 = 1,Yes,,,Only if 85 or 86 = 1,Only if fields 80 - 84 and 86 are not null,Only if fields 80 - 85 are not null,Only if field 48 = 3 or 9 ,"If 87 = 2 or 3;
if 87 = 1, then a value must be entered",No,,,Yes,,,,No,,,"If the property is being let for the first time, enter 0.","Only if 1 = 2, 4, 6, 8, 10 or 12",,,,No,"Only if 1 = 2, 4, 6, 8, 10 or 12;
or 106 = 15 - 17",No,"Only if 1 = 2, 4, 6, 8, 10 or 12",,,,No,,No,"Yes, if 45 = 2, 3 or 6",,"Yes, if 50 = 1","Only if 1 = 1, 3, 5, 7, 9 or 11",No,Yes,,,,,,,,,,Only if 1 = 1 - 4 or 9 - 12.,Only if 1 = 1 - 8.,Only if 130 is not 3,No,No,Yes,
Bulk upload format and duplicate check,All lettings,Question removed from 22/23 onwards,,Supported housing only,,Question Removed from 2020/21,,,,,,Duplicate check field,,,,,,,,Duplicate check field,,,,,,,,,,,,,,,Duplicate check field,,,,,,,,,,,,,,,,,,,Question removed from 22/23 onwards,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,Duplicate check field,,,,,,,,,,,Question removed from 22/23 onwards,Duplicate check fields,,,,Duplicate check field,General Needs lettings only,,,All lettings,General Needs lettings only,All lettings,General Needs lettings only,,,Question removed from 2020/21,Duplicate check field, “Username does not exist”. ,,,Question removed from 21/22 onwards,,Supported Housing lettings only.,,,,,,,,,,,,Affordable Rent Lettings only,Intermediate Rent Lettings only,,All lettings,All lettings,,
Bulk upload field number,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,64,65,66,67,68,69,70,71,72,73,74,75,76,77,78,79,80,81,82,83,84,85,86,87,88,89,90,91,92,93,94,95,96,97,98,99,100,101,102,103,104,105,106,107,108,109,110,111,112,113,114,115,116,117,118,119,120,121,122,123,124,125,126,127,128,129,130,131,132,133,134,
,1,,,,,,123,1,2,,6,55,54,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,
1 Question What is the letting type? [BLANK] [BLANK] Management group code If supported housing Scheme code If supported housing [BLANK] What is the tenant code? This is how you usually refer to this tenancy on your own systems Is this a starter tenancy? What is the tenancy type? If 'Other' what is the tenancy type? What is the length of the fixed-term tenancy to the nearest year? If fixed-term tenancy Age of Person 1 Age of Person 2 Age of Person 3 Age of Person 4 Age of Person 5 Age of Person 6 Age of Person 7 Age of Person 8 Gender identity of Person 1 Gender identity of Person 2 Gender identity of Person 3 Gender identity of Person 4 Gender identity of Person 5 Gender identity of Person 6 Gender identity of Person 7 Gender identity of Person 8 Person 2's relationship to lead tenant Person 3's relationship to lead tenant Person 4's relationship to lead tenant Person 5's relationship to lead tenant Person 6's relationship to lead tenant Person 7's relationship to lead tenant Person 8's relationship to lead tenant Working situation of Person 1 Working situation of Person 2 Working situation of Person 3 Working situation of Person 4 Working situation of Person 5 Working situation of Person 6 Working situation of Person 7 Working situation of Person 8 What is the lead tenant’s ethnic group? What is the lead tenant’s nationality? Does anybody in the household have links to the UK armed forces? Was the person seriously injured or ill as a result of serving in the UK armed forces? Is anybody in the household pregnant? Is the tenant likely to be receiving benefits related to housing? How much of the household's income is from Universal Credit, state pensions or benefits? How much income does the household have in total? Include any income after tax from employment, pensions, and Universal Credit Don't include National Insurance (NI) contributions and tax, housing benefit, child benefit, or council tax support Do you know the household's income? What is the tenant's main reason for the household leaving their last settled home? If 'other', what was the main reason for leaving their last settled home? [BLANK] Disabled access needs a) Fully wheelchair-accessible housing Disabled access needs b) Wheelchair access to essential rooms Disabled access needs c) Level access housing Disabled access needs f) Other disabled access needs Disabled access needs g) No disabled access needs Disabled access needs h) Don’t know Where was the household immediately before this letting? What is the local authority of the household's last settled home? Part 1 of postcode of last settled home Part 2 of postcode of last settled home Do you know the postcode of the last settled home? How long has the household continuously lived in the local authority area of the new letting? How long has the household been on the local authority waiting list for the new letting? Was the tenant homeless directly before this tenancy? Was the household given 'reasonable preference' by the local authority? Reasonable Preference They were homeless or about to lose their home (within 56 days) Reasonable Preference They were living in unsanitary, overcrowded or unsatisfactory housing Reasonable Preference They needed to move due to medical and welfare reasons (including disability) Reasonable Preference They needed to move to avoid hardship to themselves or others Reasonable Preference Don't know Was the letting made under Choice-Based Lettings (CBL)? Was the letting made under the Common Housing Register (CHR)? Was the letting made under the Common Allocation Policy (CAP)? What was the source of referral for this letting? How often does the household pay rent and other charges? What is the basic rent? What is the service charge? What is the personal service charge? What is the support charge? Total charge If this is a care home, how much does the household pay every [time period]? Does the household pay rent or other charges for the accommodation? If supported housing After the household has received any housing-related benefits, will they still need to pay basic rent and other charges? What do you expect the outstanding amount to be? What is the void or renewal day? - DD What is the void or renewal month? - MM What is the void or renewal year? - YYYY What day were Major Repairs completed on? - DD What month were Major Repairs completed on? - MM What year were Major Repairs completed on? - YY [BLANK] What day did the tenancy start? - DD What month did the tenancy start? - MM What year did the tenancy start? - YY Since becoming available, how many times has the property been previously offered? What is the property reference? This is how you usually refer to this property on your own systems How many bedrooms does the property have? If general needs What type of unit is the property? If general needs Which type of building is the property? If general needs Is the property built or adapted to wheelchair-user standards? If general needs What type was the property most recently let as? If re-let What is the reason for the property being vacant? What is the local authority of the property? If general needs Part 1 of postcode of property If general needs Part 2 of postcode of property If general needs [BLANK] Which organisation owns this property? Organisation's CORE ID Username Field Which organisation manages this property?  Organisation's CORE ID Is the person still serving in the UK armed forces? [BLANK] How often does the household receive income? Is this letting in sheltered accommodation? If supported housing Does anybody in the household have a physical or mental health condition (or other illness) expected to last 12 months or more? Vision, for example blindness or partial sight Hearing, for example deatness or partial hearing Mobility, for example walking short distances or climbing stairs Dexterity, for example lifting and carrying objects, using a keyboard Learning or understanding or concentrating Memory Mental health, for example depression or anxiety Stamina or breathing or fatigue Socially or behaviourally (e.g. associated with autism spectrum disorder (ASD) which includes Aspergers', or attention deficit hyperactivity disorder (ADHD)) Other illness or condition Is this letting a London Affordable Rent letting? Which type of Intermediate Rent is this letting? Which 'Other' type of Intermediate Rent is this letting? Has the tenant seen the DLUHC privacy notice? Is this a joint tenancy? Is this a renewal to the same tenant in the same property?
2 Values 1 - 12 1 - 999 max 13 digits 1 - 2 2 - 7 Text 1 - 99 15 - 120 or R 1 - 120 or R M, F, X or R P,C,X or R 0 - 11 1 - 19 12 - 13, 17 - 19 1 - 6 1 - 3 1, 3, 6, 9 1 - 4 0 - 99999 1 - 4 1 - 2, 4, 7 - 14, 16 - 20, 28 - 31 or 34 - 47 Text 1 or null 3- 4, 6 - 7, 9 - 10, 13 - 14, 18 - 19, 21 or 23 - 35 ONS CODE - E + 8 Digits XXX(X) XXX 1 - 2 1 - 2 or 5 - 10 2 or 5 - 11 1 or 12 1 - 3 1 or null 1 or 2 1 - 4, 7 - 10, 12 - 17 1 - 9 xxxx.xx 1 or null 1 - 3 0 - 9999 1 - 31 1 - 12 19 - 23 1 - 31 1 - 12 13 - 23 1 - 31 1 - 12 22-23 0+ 12 Digits 1 - 7 1, 2, 4 or 6 - 10 1 or 2 1 - 4 5, 6, 8 - 14, ONS CODE E + 9 digits XXX(X) XXX Up to 7 digits Username of CORE account this letting log should be assigned to Up to 7 digits 3 - 6 1 - 3 1 - 4 1 - 3 1 or null 1 - 3 Text 1 1 - 3 1 - 2
3 Can be null? No only if field 1 = 1, 3, 5, 7, 9 or 11 No Yes, if 9 is not 3 Yes, if 9 = 2, 3, 5 or 7 No Yes, if field 21, 28 and 36 are null Yes, if 22, 29 and 37 are null Yes, if 23, 30 and 38 are null Yes, if 24, 31 and 39 are null Yes, if 25, 32 and 40 are null Yes, if 26, 33 and 41 are null Yes, if 27, 34 and 42 are null No Yes, if 13, 28 and 36 are null Yes, if 14, 29 and 37 are null Yes, if 15, 30 and 38 are null Yes, if 16, 31 and 39 are null Yes, if 17, 32 and 40 are null Yes, if 18, 33 and 41 are null Yes, if 19, 34 and 42 are null Yes, if 13, 21 and 36 are null Yes, if 14, 22 and 37 are null Yes, if 15, 23 and 38 are null Yes, if 16, 24 and 39 are null Yes, if 17, 25 and 40 are null Yes, if 18, 26 and 41 are null Yes, if 19, 27 and 42 are null No Yes, if 13, 21 and 28 are null Yes, if 14, 22 and 29 are null Yes, if 15, 23 and 30 are null Yes, if 16, 24 and 31 are null Yes, if 17, 25 and 32 are null Yes, if 18, 26 and 33 are null Yes, if 19, 27 and 34 are null No Yes, must be null if 45 = 2 or 3; no, if 45 = 1, 4 or 5 No If 51 = 4 No No Yes, if 52 is not 20 Selections are ((A or B or C)) - ((A, or B or C and F)) - ((F)) - ((G)) - ((H)) No Yes, if 65 = 1 Yes, if 63 and 64 contain full and valid entries No If 69 = 1, select at least one of the 5 categories; If 69 = 2 or 3, then Null. No Only if 85 or 86 = 1 Yes Only if 85 or 86 = 1 Only if fields 80 - 84 and 86 are not null Only if fields 80 - 85 are not null Only if field 48 = 3 or 9 If 87 = 2 or 3; if 87 = 1, then a value must be entered No Yes No If the property is being let for the first time, enter 0. Only if 1 = 2, 4, 6, 8, 10 or 12 No Only if 1 = 2, 4, 6, 8, 10 or 12; or 106 = 15 - 17 No Only if 1 = 2, 4, 6, 8, 10 or 12 No No Yes, if 45 = 2, 3 or 6 Yes, if 50 = 1 Only if 1 = 1, 3, 5, 7, 9 or 11 No Yes Only if 1 = 1 - 4 or 9 - 12. Only if 1 = 1 - 8. Only if 130 is not 3 No No Yes
4 Bulk upload format and duplicate check All lettings Question removed from 22/23 onwards Supported housing only Question Removed from 2020/21 Duplicate check field Duplicate check field Duplicate check field Question removed from 22/23 onwards Duplicate check field Question removed from 22/23 onwards Duplicate check fields Duplicate check field General Needs lettings only All lettings General Needs lettings only All lettings General Needs lettings only Question removed from 2020/21 Duplicate check field “Username does not exist”. Question removed from 21/22 onwards Supported Housing lettings only. Affordable Rent Lettings only Intermediate Rent Lettings only All lettings All lettings
5 Bulk upload field number 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134
6 1 123 1 2 6 55 54

716
spec/services/bulk_upload/lettings/row_parser_spec.rb
Unescape View File

@ -3,77 +3,733 @@ require "rails_helper"
RSpec.describe BulkUpload::Lettings::RowParser do
subject(:parser) { described_class.new(attributes) }
let(:now) { Time.zone.today }
let(:attributes) { { bulk_upload: } }
let(:bulk_upload) { create(:bulk_upload, :lettings, user:) }
let(:user) { create(:user, organisation: owning_org) }
let(:owning_org) { create(:organisation) }
let(:managing_org) { create(:organisation) }
let(:setup_section_params) do
{
bulk_upload:,
field_1: "2",
field_111: owning_org.old_visible_id,
field_113: managing_org.old_visible_id,
field_96: now.day.to_s,
field_97: now.month.to_s,
field_98: now.strftime("%g"),
field_134: "2",
}
end
around do |example|
FormHandler.instance.use_real_forms!
example.run
FormHandler.instance.use_fake_forms!
end
describe "validations" do
before do
stub_request(:get, /api.postcodes.io/)
.to_return(status: 200, body: "{\"status\":200,\"result\":{\"admin_district\":\"Manchester\", \"codes\":{\"admin_district\": \"E08000003\"}}}", headers: {})
parser.valid?
end
describe "field_1" do
describe "#valid?" do
context "when calling the method multiple times" do
let(:attributes) { { bulk_upload:, field_134: 2 } }
it "does not add keep adding errors to the pile" do
expect { parser.valid? }.not_to change(parser.errors, :count)
end
end
context "when valid row" do
let(:attributes) do
{
bulk_upload:,
field_1: "1",
field_4: "1",
field_7: "123",
field_96: now.day.to_s,
field_97: now.month.to_s,
field_98: now.strftime("%g"),
field_108: "EC1N",
field_109: "2TD",
field_111: owning_org.old_visible_id,
field_113: managing_org.old_visible_id,
field_130: "1",
field_134: "2",
field_102: "2",
field_103: "1",
field_104: "1",
field_101: "1",
field_133: "2",
field_8: "1",
field_9: "2",
field_132: "1",
field_12: "42",
field_13: "41",
field_14: "20",
field_15: "18",
field_16: "16",
field_17: "14",
field_18: "12",
field_19: "20",
field_20: "F",
field_21: "M",
field_22: "F",
field_23: "M",
field_24: "F",
field_25: "M",
field_26: "F",
field_27: "M",
field_43: "17",
field_44: "18",
field_28: "P",
field_29: "C",
field_30: "X",
field_31: "R",
field_32: "C",
field_33: "C",
field_34: "X",
field_35: "1",
field_36: "2",
field_37: "6",
field_38: "7",
field_39: "8",
field_40: "9",
field_41: "0",
field_42: "10",
field_45: "1",
field_114: "4",
field_46: "1",
field_47: "1",
field_118: "2",
field_66: "5",
field_67: "2",
field_52: "31",
field_61: "3",
field_68: "12",
field_65: "1",
field_63: "EC1N",
field_64: "2TD",
field_69: "1",
field_70: "1",
field_71: "",
field_72: "1",
field_73: "",
field_74: "",
field_75: "1",
field_76: "2",
field_77: "2",
field_78: "2",
field_51: "1",
field_50: "2000",
field_116: "2",
field_48: "1",
field_49: "1",
field_79: "4",
field_80: "1234.56",
field_87: "1",
field_88: "234.56",
}
end
it "returns true" do
expect(parser).to be_valid
end
it "instantiates a log with everything completed", aggregate_failures: true do
questions = parser.send(:questions).reject do |q|
parser.send(:log).optional_fields.include?(q.id) || q.completed?(parser.send(:log))
end
expect(questions.map(&:id).size).to eq(0)
expect(questions.map(&:id)).to eql([])
end
end
end
describe "#field_1" do
context "when null" do
let(:attributes) { { field_1: nil } }
let(:attributes) { { bulk_upload:, field_1: nil } }
it "returns an error" do
expect(parser.errors[:field_1]).to be_present
end
end
context "when incorrect data type" do
let(:attributes) { { bulk_upload:, field_1: "foo" } }
it "returns an error" do
expect(parser.errors).to include(:field_1)
expect(parser.errors[:field_1]).to be_present
end
end
context "when outside permited range" do
let(:attributes) { { field_1: "13" } }
context "when unpermitted value" do
let(:attributes) { { bulk_upload:, field_1: "101" } }
it "returns an error" do
expect(parser.errors).to include(:field_1)
expect(parser.errors[:field_1]).to be_present
end
end
context "when valid" do
let(:attributes) { { field_1: 1 } }
let(:attributes) { { bulk_upload:, field_1: "1" } }
it "is valid" do
expect(parser.errors).not_to include(:field_1)
it "does not return any errors" do
expect(parser.errors[:field_1]).to be_blank
end
end
end
describe "field_4" do
context "when text" do
let(:attributes) { { field_4: "R" } }
describe "#field_4" do
context "when nullable not permitted" do
let(:attributes) { { bulk_upload:, field_1: "2", field_4: nil } }
it "is not valid" do
expect(parser.errors).to include(:field_4)
it "cannot be nulled" do
expect(parser.errors[:field_4]).to be_present
end
end
context "when valid" do
let(:attributes) { { field_4: "3" } }
context "when nullable permitted" do
let(:attributes) { { bulk_upload:, field_1: "1", field_4: nil } }
it "can be nulled" do
expect(parser.errors[:field_4]).to be_blank
end
end
context "when matching scheme cannot be found" do
let(:attributes) { { bulk_upload:, field_1: "1", field_4: "123" } }
xit "returns an error" do
expect(parser.errors[:field_4]).to be_present
end
end
end
describe "#field_7" do
context "when null" do
let(:attributes) { { bulk_upload:, field_7: nil } }
it "is valid" do
expect(parser.errors).not_to include(:field_4)
xit "returns an error" do
expect(parser.errors[:field_7]).to be_present
end
end
end
context "when allowed to be null" do
let(:attributes) { { field_1: "2", field_4: "" } }
describe "#field_10" do
context "when field_9 is 3 aka other" do
let(:attributes) { { bulk_upload:, field_9: "3" } }
it "is valid" do
expect(parser.errors).not_to include(:field_4)
xit "returns an error" do
expect(parser.errors[:field_10]).to be_present
end
end
end
context "when not allowed to be null" do
let(:attributes) { { field_1: "3", field_4: "" } }
describe "fields 96, 97, 98 => startdate" do
context "when any one of these fields is blank" do
let(:attributes) { { bulk_upload:, field_96: nil, field_97: nil, field_98: nil } }
it "is not valid" do
expect(parser.errors).to include(:field_4)
it "returns an error" do
expect(parser.errors[:field_96]).to be_present
expect(parser.errors[:field_97]).to be_present
expect(parser.errors[:field_98]).to be_present
end
end
end
describe "#field_134" do
context "when not a possible value" do
let(:attributes) { { field_134: "3" } }
context "when an unpermitted value" do
let(:attributes) { { bulk_upload:, field_134: 3 } }
it "has errors on the field" do
expect(parser.errors[:field_134]).to be_present
end
end
end
describe "#field_103" do
context "when null" do
let(:attributes) { setup_section_params.merge({ field_103: nil }) }
it "returns an error" do
expect(parser.errors[:field_103]).to be_present
end
end
context "when unpermitted values" do
let(:attributes) { setup_section_params.merge({ field_103: "4" }) }
it "returns an error" do
expect(parser.errors[:field_103]).to be_present
end
end
end
end
describe "#log" do
describe "#cbl" do
context "when field_75 is yes ie 1" do
let(:attributes) { { bulk_upload:, field_75: 1 } }
it "sets value to 1" do
expect(parser.log.cbl).to be(1)
end
end
context "when field_75 is no ie 2" do
let(:attributes) { { bulk_upload:, field_75: 2 } }
it "sets value to 0" do
expect(parser.log.cbl).to be(0)
end
end
end
describe "#chr" do
context "when field_76 is yes ie 1" do
let(:attributes) { { bulk_upload:, field_76: 1 } }
it "sets value to 1" do
expect(parser.log.chr).to be(1)
end
end
context "when field_76 is no ie 2" do
let(:attributes) { { bulk_upload:, field_76: 2 } }
it "sets value to 0" do
expect(parser.log.chr).to be(0)
end
end
end
describe "#cap" do
context "when field_77 is yes ie 1" do
let(:attributes) { { bulk_upload:, field_77: 1 } }
it "sets value to 1" do
expect(parser.log.cap).to be(1)
end
end
context "when field_77 is no ie 2" do
let(:attributes) { { bulk_upload:, field_77: 2 } }
it "sets value to 0" do
expect(parser.log.cap).to be(0)
end
end
end
describe "#letting_allocation_unknown" do
context "when field_75, 76, 77 are no ie 2" do
let(:attributes) { { bulk_upload:, field_75: 2, field_76: 2, field_77: 2 } }
it "sets value to 1" do
expect(parser.log.letting_allocation_unknown).to be(1)
end
end
context "when any one of field_75, 76, 77 is yes ie 1" do
let(:attributes) { { bulk_upload:, field_75: 1 } }
it "sets value to 0" do
expect(parser.log.letting_allocation_unknown).to be(0)
end
end
end
describe "#renewal" do
context "when field_134 is no ie 2" do
let(:attributes) { { bulk_upload:, field_134: 2 } }
it "sets value to 0" do
expect(parser.log.renewal).to eq(0)
end
end
context "when field_134 is null but rsnvac/field_116 is 14" do
let(:attributes) { { bulk_upload:, field_134: "", field_116: "14" } }
it "sets renewal to 1" do
expect(parser.log.renewal).to eq(1)
end
end
end
describe "#sexN fields" do
let(:attributes) do
{
bulk_upload:,
field_20: "F",
field_21: "M",
field_22: "X",
field_23: "R",
field_24: "F",
field_25: "M",
field_26: "X",
field_27: "R",
}
end
it "sets value from correct mapping" do
expect(parser.log.sex1).to eql("F")
expect(parser.log.sex2).to eql("M")
expect(parser.log.sex3).to eql("X")
expect(parser.log.sex4).to eql("R")
expect(parser.log.sex5).to eql("F")
expect(parser.log.sex6).to eql("M")
expect(parser.log.sex7).to eql("X")
expect(parser.log.sex8).to eql("R")
end
end
describe "#ecstatN fields" do
let(:attributes) do
{
bulk_upload:,
field_35: "1",
field_36: "2",
field_37: "6",
field_38: "7",
field_39: "8",
field_40: "9",
field_41: "0",
field_42: "10",
}
end
it "sets value from correct mapping", aggregate_failures: true do
expect(parser.log.ecstat1).to eq(1)
expect(parser.log.ecstat2).to eq(2)
expect(parser.log.ecstat3).to eq(6)
expect(parser.log.ecstat4).to eq(7)
expect(parser.log.ecstat5).to eq(8)
expect(parser.log.ecstat6).to eq(9)
expect(parser.log.ecstat7).to eq(0)
expect(parser.log.ecstat8).to eq(10)
end
end
describe "#relatN fields" do
let(:attributes) do
{
bulk_upload:,
field_28: "P",
field_29: "C",
field_30: "X",
field_31: "R",
field_32: "P",
field_33: "C",
field_34: "X",
}
end
it "sets value from correct mapping", aggregate_failures: true do
expect(parser.log.relat2).to eq("P")
expect(parser.log.relat3).to eq("C")
expect(parser.log.relat4).to eq("X")
expect(parser.log.relat5).to eq("R")
expect(parser.log.relat6).to eq("P")
expect(parser.log.relat7).to eq("C")
expect(parser.log.relat8).to eq("X")
end
end
describe "#net_income_known" do
let(:attributes) { { bulk_upload:, field_51: "1" } }
it "sets value from correct mapping" do
expect(parser.log.net_income_known).to eq(0)
end
end
describe "#unitletas" do
let(:attributes) { { bulk_upload:, field_105: "1" } }
it "sets value from correct mapping" do
expect(parser.log.unitletas).to eq(1)
end
end
describe "#rsnvac" do
let(:attributes) { { bulk_upload:, field_106: "5" } }
it "sets value from correct mapping" do
expect(parser.log.rsnvac).to eq(5)
end
end
describe "#sheltered" do
let(:attributes) { { bulk_upload:, field_117: "1" } }
it "sets value from correct mapping" do
expect(parser.log.sheltered).to eq(1)
end
end
describe "illness fields" do
mapping = [
{ attribute: :illness_type_1, field: :field_119 },
{ attribute: :illness_type_2, field: :field_120 },
{ attribute: :illness_type_3, field: :field_121 },
{ attribute: :illness_type_4, field: :field_122 },
{ attribute: :illness_type_5, field: :field_123 },
{ attribute: :illness_type_6, field: :field_124 },
{ attribute: :illness_type_7, field: :field_125 },
{ attribute: :illness_type_8, field: :field_126 },
{ attribute: :illness_type_9, field: :field_127 },
{ attribute: :illness_type_10, field: :field_128 },
]
mapping.each do |hash|
describe "##{hash[:attribute]}" do
context "when yes" do
let(:attributes) { { bulk_upload:, hash[:field] => "1" } }
it "sets value from correct mapping" do
expect(parser.log.public_send(hash[:attribute])).to eq(1)
end
end
context "when no" do
let(:attributes) { { bulk_upload:, hash[:field] => "" } }
it "sets value from correct mapping" do
expect(parser.log.public_send(hash[:attribute])).to be_nil
end
end
end
end
end
describe "#irproduct_other" do
let(:attributes) { { bulk_upload:, field_131: "some other product" } }
it "sets value to given free text string" do
expect(parser.log.irproduct_other).to eql("some other product")
end
end
describe "#tenancyother" do
let(:attributes) { { bulk_upload:, field_10: "some other tenancy" } }
it "sets value to given free text string" do
expect(parser.log.tenancyother).to eql("some other tenancy")
end
end
describe "#tenancylength" do
let(:attributes) { { bulk_upload:, field_11: "2" } }
it "sets value to given free text string" do
expect(parser.log.tenancylength).to eq(2)
end
end
describe "#earnings" do
let(:attributes) { { bulk_upload:, field_50: "104.50" } }
it "rounds to the nearest whole pound" do
expect(parser.log.earnings).to eq(105)
end
end
describe "#reasonother" do
let(:attributes) { { bulk_upload:, field_53: "some other reason" } }
it "sets value to given free text string" do
expect(parser.log.reasonother).to eql("some other reason")
end
end
describe "#ppcodenk" do
let(:attributes) { { bulk_upload:, field_65: "2" } }
it "sets correct value from mapping" do
expect(parser.log.ppcodenk).to eq(0)
end
end
describe "#household_charge" do
let(:attributes) { { bulk_upload:, field_86: "1" } }
it "sets correct value from mapping" do
expect(parser.log.household_charge).to eq(1)
end
end
describe "#chcharge" do
let(:attributes) { { bulk_upload:, field_85: "123.45" } }
it "sets value given" do
expect(parser.log.chcharge).to eq(123.45)
end
end
describe "#tcharge" do
let(:attributes) { { bulk_upload:, field_84: "123.45" } }
it "sets value given" do
expect(parser.log.tcharge).to eq(123.45)
end
end
describe "#supcharg" do
let(:attributes) { { bulk_upload:, field_83: "123.45" } }
it "sets value given" do
expect(parser.log.supcharg).to eq(123.45)
end
end
describe "#pscharge" do
let(:attributes) { { bulk_upload:, field_82: "123.45" } }
it "sets value given" do
expect(parser.log.pscharge).to eq(123.45)
end
end
describe "#scharge" do
let(:attributes) { { bulk_upload:, field_81: "123.45" } }
it "sets value given" do
expect(parser.log.scharge).to eq(123.45)
end
end
describe "#offered" do
let(:attributes) { { bulk_upload:, field_99: "3" } }
it "sets value given" do
expect(parser.log.offered).to eq(3)
end
end
describe "#propcode" do
let(:attributes) { { bulk_upload:, field_100: "abc123" } }
it "sets value given" do
expect(parser.log.propcode).to eq("abc123")
end
end
describe "#mrcdate" do
let(:attributes) { { bulk_upload:, field_92: "13", field_93: "12", field_94: "22" } }
it "sets value given" do
expect(parser.log.mrcdate).to eq(Date.new(2022, 12, 13))
end
end
describe "#majorrepairs" do
context "when mrcdate given" do
let(:attributes) { { bulk_upload:, field_92: "13", field_93: "12", field_94: "22" } }
it "sets #majorrepairs to 1" do
expect(parser.log.majorrepairs).to eq(1)
end
end
context "when mrcdate not given" do
let(:attributes) { { bulk_upload:, field_92: "", field_93: "", field_94: "" } }
it "sets #majorrepairs to 0" do
expect(parser.log.majorrepairs).to eq(0)
end
end
end
describe "#voiddate" do
let(:attributes) { { bulk_upload:, field_89: "13", field_90: "12", field_91: "22" } }
it "sets value given" do
expect(parser.log.voiddate).to eq(Date.new(2022, 12, 13))
end
end
describe "#startdate" do
let(:attributes) { { bulk_upload:, field_96: now.day.to_s, field_97: now.month.to_s, field_98: now.strftime("%g") } }
it "sets value given" do
expect(parser.log.startdate).to eq(now)
end
end
describe "#postcode_full" do
let(:attributes) { { bulk_upload:, field_108: " EC1N ", field_109: " 2TD " } }
it "strips whitespace" do
expect(parser.log.postcode_full).to eql("EC1N 2TD")
end
end
describe "#la" do
let(:attributes) { { bulk_upload:, field_107: "E07000223" } }
it "sets to given value" do
expect(parser.log.la).to eql("E07000223")
end
end
describe "#prevloc" do
let(:attributes) { { bulk_upload:, field_62: "E07000223" } }
it "sets to given value" do
expect(parser.log.prevloc).to eql("E07000223")
end
end
describe "#previous_la_known" do
context "when known" do
let(:attributes) { { bulk_upload:, field_62: "E07000223" } }
it "sets to 1" do
expect(parser.log.previous_la_known).to eq(1)
end
end
context "when not known" do
let(:attributes) { { bulk_upload:, field_62: "" } }
it "sets to 0" do
expect(parser.log.previous_la_known).to eq(0)
end
end
end
describe "#first_time_property_let_as_social_housing" do
context "when field_106 is 15, 16, or 17" do
let(:attributes) { { bulk_upload:, field_106: %w[15 16 17].sample } }
it "is not valid" do
expect(parser.errors).to include(:field_134)
it "sets to 1" do
expect(parser.log.first_time_property_let_as_social_housing).to eq(1)
end
end
end

2
spec/services/bulk_upload/lettings/validator_spec.rb
Unescape View File

@ -30,7 +30,7 @@ RSpec.describe BulkUpload::Lettings::Validator do
end
context "when a valid csv" do
let(:path) { file_fixture("2021_22_lettings_bulk_upload.csv") }
let(:path) { file_fixture("2022_23_lettings_bulk_upload.csv") }
it do
validator.call

4
spec/services/bulk_upload/processor_spec.rb
Unescape View File

@ -11,7 +11,7 @@ RSpec.describe BulkUpload::Processor do
instance_double(
BulkUpload::Downloader,
call: nil,
path: file_fixture("2021_22_lettings_bulk_upload.csv"),
path: file_fixture("2022_23_lettings_bulk_upload.csv"),
delete_local_file!: nil,
)
end
@ -19,7 +19,7 @@ RSpec.describe BulkUpload::Processor do
it "persist the validation errors" do
allow(BulkUpload::Downloader).to receive(:new).with(bulk_upload:).and_return(mock_downloader)
expect { processor.call }.to change(BulkUploadError, :count).by(9)
expect { processor.call }.to change(BulkUploadError, :count)
end
it "deletes the local file afterwards" do

Write Preview
Loading…
Cancel
Save