Version - 2.0.5

• ❇️ New
•  Implemented OAuth login for Salesforce Data Sources (ref 4983)
•  Implement OEM salesforce database driver (ref 4885)
•  Added a user interface guide (ref 4850)
• 552 Added open source licences to the Help page (ref 4856)
• #405 Allow rows to be tagged with categories (ref 4669)
• #385 Added Import and Export feature for Projects (ref 4487)
• #537 Added view only mode to the Edit Data Entity screen for users without permissions (ref 4826)
•  Added maintenance mode for orderly updates (ref 4533)
•  Added SoapClient to allow for iData scripts to interact with SOAP API's (ref 4705)
•  Added Host.XmlToJson and Host.JsonToXml that allow converstion between Json and XML (ref 4707)
• #491 Added Host.GetMachineID() to get machine ID in scripts (ref 4714)
• 🔀 Changed
• 577 Update iData Reference Manual regarding TagAsEmail().WithSimpleValidation() (ref 4894)
•  A valid server certificate is now required for MSSql data sources. To proceed without a valid certificate, the “Trust Server Certificate” flag under a selected data source can be enabled. (ref 4933)
• #585 Importing projects will now redirect users to update datasources. Stages are prevented from running until all datasources are valid.  (ref 4928)
•  Reduced the load time of Data Entities when creating a stage (ref 4844)
•  Refactored username lookups for datatables (ref 4867)
• #420 The transform script output now has specific error text for attempting to use OnNthRow while no unique keys are set (ref 4865)
• #549 Data sources are now individually licensed. An updated license file will be required. (ref 4857)
• #417 Updated deprecated RndPatterns method in default transform script (ref 4864)
•  Updated SalesForce documentation in the iData Reference Manual (ref 4843)
• #534,#536 Users with user management permission allowed to enable 2FA (ref 4810)
•  Clicking "Close" after ingesting a table now takes the user directly to entity creation, letting them create an entity from the ingested table (ref 4735)
• ☑️ Fixed
• 640 Fixed an error that occured when submitting an invalid server name (ref 5113)
• 642 The new Stage screen no longer throws an error when a licence is missing (ref 5105)
• #634 Fixed issue with parameter box on the schedules page not loading correctly (ref 5097)
• 586 Fixed authorisation error on Job Queue page (ref 5095)
• #539 Fixed issue with schedules page showing schedules from other projects  (ref 5083)
•  Fixed an issue that caused Excel files to become currupt in iData 2.0 (ref 4666)
• 619 Added bool to default MSSqlWhitelist (ref 5060)
• #602 Added reminder to update connection string (step 5) in error message (step 6). (ref Story 5008)
• #608 Solved null reference exception when creating Data Gen stage with a Salesforce entity (ref 5044)
• #594 Fixed error when clicking on an "In Progress" report (ref 4960)
• #606 Fixed running data generation on salesforce (ref 5016)
• #576 Fixed iData reference manual typo in TagAsDomain (ref 4895)
• #575 Updated iData reference manual "Manually tagging" reference (ref 4896)
• #587 Fixed issue running stages after migration (ref 4937)
• #565 Fixed issue where the edit entity page was returning a 400 error (ref 4903)
• 566 Resolved the "Method GetScriptArguments threw ArgumentNullException: Value cannot be null. (Parameter 'g')" error message (ref 4902)
•  Stages of imported projects can now be executed (ref 4913)
• #556 Fixed long banner text causes layout issues (ref 4870)
• #249 Files uploaded to lookup folder are automatically made accessible from scripts (ref 4854)
• #415 Implemented RndGuid() when primary key type is uniqueidentifier for default transform scripts (ref 4863)
• #546,#547,#558 Fixed issues in creating non-word stage name  (ref 4829)
•  Data Entity dependencies are now automatically created to prevent errors from missing references (ref 4845)
• #547 Fixed an issue where stage names with spaces would not generate scripts (ref 4855)
• #503 iData now handles controlled shutdowns gracefully (ref 4755)
• #279 Fixed issue where the newly created project is not selected by default (ref 4798)
• #559 Ensure all database migration steps are recorded to avoid database upgrade warning message (ref 4879)
• #531 Fixed issue with log download API pointing to the wrong location (ref 4808)
• #533 Fixed to display role names for users with the Delete Role permission only (ref 4809)
• #543 Fixed restore user confirm message (ref 4828)
• #535 Updated roles, permissions, and project permissions to prevent users from getting more access by creating another account (ref 4811)
• #545 Fixed an issue with Profile Results that prevented List View from displaying when View All Tables was selected in the filter (ref 4819)
• #357 User account search now returning results (ref 4432)
• #485 Source Entities no longer fail to load when editing mappings for Single Mapping stages (ref 4698)
• #512 Fixed file locks when writing to file in scripts (ref 4779)
• #518 Add permission checking in File browser (ref 4792)
• #538 Fixed view only permission closing script editor issue (ref 4813)
• #495 Re-visited users being logged out when marked as inactive (ref 3999)
•  Similar records no longer skips a row after matching a group (ref 4458)
• #484 Users are now forced to select a column in Similar Records stage (ref 4697)
• #492 Apply consistent row ordering in reports (ref 4737)
• #493 TagAsCompanyEmail tag added to differentiate between different types of emails. Appropriate email now generated according to selected email tag (ref 4739)
• #495 Fixed issue where inactive users were being sent to the email confirmation screen when they attempt to log in (ref 4743)
• #523 Allow admin role users delete other users project (ref 4794)
• #526 Datatables in modals no longer have the same fixed height as datatables on the page (ref 4804)
• #530 Fixed typo on 2FA setting page (ref 4807)

Version - 2.0.4

• ❇️ New
•  Added a page before ingesting a CSV file into a database that describes what the action will do 
•  Added a "Create Folder" button on CSV Datasource creation modal. This allows new folders to be used as bases for Datasources 
•  Added a "New Folder" button for Files 
•  Added a "Browse and Upload Files" option for CSV Datasources 
•  Added option to ingest CSV file into a database via the action menu 
•  Files and folders can now be downloaded via the actions menu 
•  Added ability to categories records and view them in the report 
• 🔀 Changed
•  Removed SetColumnToChar due to SQL incompatibility and JSON object issues 
•  #442 update EULA in installer 
•  Removed legacy Host.NewDbCompare 
•  Depreciated SearchGroups for SearchGroup while an issue with multiple runs is resolved 
•  #487 Default data generation scripts for number columns will now generate data relevant to that field type instead of just integers 
•  Moved the "Upload Folder" button for CSV Datasources to the top of the modal 
•  The folder rename dialog in Files no longer requires you to manually remove the backslash and is now titled "Renamed Folder" 
•  Changed how client secrets are stored, this may require creating a new client secret for the worker via the Administration -> Settings -> Worker Settings menu 
•  #479 iData virtual file system storage and performance improvements 
•  Csv datasource re-enabled 
• ☑️ Fixed
•  #490 Fixed issue involving characters in file paths 
•  #469 Fixed issue where execution time was not taking UTC into account 
•  #454 Fixed issue where data type mappings are not applied correctly when comparing a CSV to a table 
•  ETL tooltips now show the names of each row state 
•  Fixed issue with updating progress bars that caused exceptions in the log and the progress bar to not show in the UI 
•  Fixed issue where DBMigrator was not accepting some characters as symbols 
•  #463 Fixed ussue on Schedule page where parameters with checkboxes where not working correctly 
•  #450 Fixed grammar in the limited results text for the report details page 
•  #480 The new CSV Datasource dialog now fits the screen 
•  #464 Permission are now all visible without scrolling 
•  #382 Fixed similar records skipping rows when two groups are close together 
•  #497 Fixed the select data source screen not showing data sources 
•  #506 Fixed edit CSV Datasource no responses issue 
•  #514 Fixed issue where Host.ExitCode was being ignored. Negative values will now cause the script to fail 
•  #527 Fixed new CSV Datasource page upload folder button disappear issue 
•  #529 Fixed stage page and others pages that were not displaying results because the user who created the item had been deleted 

Version - 2.0.3

• 🔀 Changed
•  Disabled csv datasource until #479 resolved 

Version - 2.0.2

• ❇️ New
•  On application launch systeminfo.txt is written to ProgramData\IntelligentDeliverySolutions\iData 
• 🔀 Changed
•  iDataConsole now uses the same encryption as iDataSaaS 
•  #438 Updated 2FA warning message on users page to be more readable 
•  #446 Auto focus username field on login screen 
• ☑️ Fixed
•  No longer shows UpdateProgress exception in script logs when a job ends prematurely 
•  Fixed tooltips on long report values that overflow in Similar Records 
•  A CSV with a duplicate header no longer causes an unhandled exception 
•  #384 Fixed issue where users without CreateProject permission didn't have breadcrumbs 
•  #423 Fixed issue where script path would hide script editor controls 
•  #444 Fixed issue with incorrect filters showing on the report page 
•  #453 Cancelling a job no longer throws a stack trace to the console output 
•  #462 Fixed Typo on Log Settings Screen 
•  #468 Fixed CSV Download issue 
•  #470 Fixed issue allow editing entities with same name on different datasource 
•  #471 Fixed where cancelling a stage generates an error and fails to stop the stage from running 

Version - 2.0.1

• ❇️ New
•  Web user interface 
•  Integrated editor 
•  Datasource and Data entity management through the User interface 
•  Manage and update stages through the user interface (Replacing wizards) 
•  Schedule execution 
•  Interactive reports and drilldown 
•  Projects to organize work and users 
•  Run multiple jobs in parallel 
•  Auto logout user when the user is deleted or has permission change 
• 🔀 Changed
•  Removed support for legacy configuration files 
•  Removed support for legacy wizards. Host.HeadlessWizard, Host.NewWizard, Host.MessageBoxYesNo, Host.MessageBoxWarning have been removed 
•  Host.Exec is deprecated, use javascripts import/export statements. Find out more here: https://javascript.info/import-export 
•  EnableNewColumns() on text writers has been replaced with PreventNewColumns() 
•  Removed support for RemoveDuplicates transform 
•  Updated the UI so only one column can be selected for grouping on SimilarRecords 
• ☑️ Fixed
•  #261 & #231 Fixed sorting by name on project and stage screens 
•  #260 Changes to stages through the UI overwrite manual changes to scripts 
•  #254 Unhandled data types now fails gracefully, however further change will be made in #259 
•  #249 Manually adding a lookup file updates lookups.js to support code completion in the editor 
•  #248 Renaming a lookup file removes contents 
•  #246 Users with execute rights don't have to be in the admin group 
•  #245 Incorrect version number returned from command line 
•  #244 RndIntBetween added for generating inclusive numbers, help for RndIntRange updated to state it generates numbers excluding the end value 
•  #243 Set generate 0 or 1 values for bit fields 
•  #242 ConvertToText now takes requested region and culture into account when formatting dates 
•  #239 Column names with apostrophe's are now handled correctly 
•  #237 #228 #227 #226 #220 Cosmetic layout fixes 
•  #233 Exported results location shown with link 
•  #232 Incorrect number of pages in results 
•  #230 Re-ordered profiling list view filters into logical groups rather than alphabetically 
•  #229 Increased profile results size limits to handle larger tables 
•  #213 Favicon now displayed on login screen 
•  #184 Reports and stages can be deleted to manage database size 
•  #181 Running jobs can be cancelled 
•  #178 Timeout profiling large tables 
•  #168 Entities can be added or ammended after creation 
•  #241 Fixed Inconsistent timezone on Job Progress 
•  #224 Removed references to wizards on some screens 
•  #235 LetsEncrypt certificates can be requested via to the UI 
•  #256 Solved issue with Intermittent failure of scheduled jobs 
•  #255 Action button for schedule missing under some circumstances 
•  #263 MakeRndCardCols returns an error 
•  #253 rename GetDataEntity to GetDataEntityConfig for clearity and implement RndRecordFrom on DBQuery 
•  #259 Creating stage where entities have foreign keys to other ommited entities cases error 500 
•  #262 Error using 'Run again' from job progress or run history 
•  #276 The iData worker is now setup on application start if not already configured 
•  #274/#275 Fixed an issue where the admin role was not being properly checked for when running a script 
•  Resolved a issue when running a job "Issuer name does not match authority" 
•  #269 iData currently does not support MSSQL Spatial data types such as Geography 
•  #269 Skip unreadable data types such as spatial 
•  #300 Fixed link to job in the notification Email from showing an error when clicked 
•  #291 Fixed an issue where CSV entity names containing spaces would generate bad variable names 
•  #288 Resolved an issue when deleting a stage 
•  #297 Solved issue with uploading licence 
•  #343/#344 Fixed various styling and css issues 
•  #326 Solved issue where not all profilers where shown when hitting the 150 column limit 
•  #343/#344/338/339 Fixed various styling and css issues 
•  #323 Fixed AutoMapping CSV to SQL for Wide Table 
•  #355 Solved issue with referential integrity and data generation 
•  #356 Improved register validation to prevent User Enumeration 
•  #317 Set Similar Record modal width to prevent overflow of mapping container 
•  #318 Documents should no longer be incorrectly translated by the browser 
•  #347 Fixed an issue where new user accounts weren't being activated 
•  #309 Removed exec from user help commands 
•  #367 Fixed typo on LetsEncrypt screen along with adding a domain field and some instruction on how to change it 
•  #380 Added Two Factor Authentication 
•  #381 Added "Use Domain in Credentials" to email settings screen to allow for Gmail credentials to authenticate correctly 
•  #386 Fixed Number of Columns Compared counter not reporting anything 
•  #387 Fixed issue running stages with long names 
•  #396 Fixed CSV to CSV data mappings for strings 
•  #397 Fixed 500 error when opening a comparison report with mapped columns 
•  #400 By default we now limit the number of rows reported in each ETL and Comparison report and increased timeout on query to allow for bigger reports to be loaded 
•  #410 Resolved issue where deleting a report that is being processed courses an exception 
•  #412 Fixed 500 Error on the Settings screen 
•  #424 Fixed incorrect reference to step 4 in getting started guide 
•  #426 Updated formating on stage completion emails 
•  #432 Users are now prompted to login after registration. New Users get a welcome email 
•  #433 Fixed issue with apostrophe in CSV column name breaks scripts 
•  #449 Uploading a file no longer creates a folder with the same name 
•  #452 Encoded modal header titles before displaying them to prevent script injection 
•  #457 Fixed issue with default csv options that caused issues reading headers 
• ⚠️ Known Issues
•  #257 Order of source and target rows can swap depending on row status 
•  #252 ForEachRowFrom can't be used multiple times for the same OpenQuery 
•  #250 & #251 Generating data for tables with identity columns 
•  #183 Job stopped during processing 
•  Db Migrator always shows a warning message 
•  Unclear warning message when row size exceeded 
•  user search returns no users 
•  ignore values on similar records causes duplicates to be missed 
•  Similar records tooltip appears in wrong place 
•  Similar records occasionally missing expected results 
•  Restoring a db and ProgramData to a new server causes issues 
•  User with View Only permissions has no breadcrumb menus when viewing a report 
•  #368 Value cannot be null when using OnRndRows method on column that accepts null 
•  #401 script log updates are added to the current page even when its not the latest page 
•  #406 project owner not listed in project permissions 
•  Skipping all records seems to cause an issue 
•  #421 report columns don't line up with headers on tables with a large number of columns 
•  #422 need to scroll to see popup menu 
•  #423 Editing a script with a long path causes save and close buttons to be partially hiden 
•  #426 alert notification email message needs tidying up 
•  #436 Internittent timeout on results screen and error sending progress to source 
•  #439 job status intermittently fails to go to a green tick 
•  short columns with long names causes poor formatting in reports 

Version - 1.8.1

• ❇️ New
•  new data generation method GetRndRecord can be used to select multiple fields from a randomly selected lookup record 
•  values can be set using a 'Get' from another column when using SetText, SetInt etc 
•  RndRange transforms can now accept range parameters using a 'Get' from other columns 
•  values can be assigned using a general 'Set' that uses the data type of the parameter 
•  added RndDateTimeRange which returns a random date and time between the specified values 
•  AddressValidationAPI has an option to refresh the cache once the records are older than a configurable amount of days 
•  added request logging to AddressValidationAPI using LogRequestContaining config setting 
•  AddressValidationAPI cache is now encrypted 
•  MakeRndPersonCols when used for obfuscating guarantees a generated name will be different from original name 
•  Domains for generated email addresses have been updated to avoid clashing with real email addresses 
•  Limited SetMaxUniqueValues(int) to 100,000 to ensure stability when profiling 
•  Added ReplaceControlCharacters to remove control characters. 
•  Added ReplaceMultipleSpaces to replace multiple spaces. 
•  Added ReplaceBlanksWithNulls to replace blanks to Null. 
•  ConvertToText can now support non-numeric format specificer and culture code. 
•  Added data type validation to Csv readers when types are defined by the user, with functions to handle validation errors, OnReadErrorContinue() and OnReadErrorAbort() 
•  Allow scripts to prevent automatic failure of a row after validating an email or address and implement their own handling of the issue including retrieving the error message. See IValidateCondition functions 
• 🔀 Changed
•  IMPORTANT: Selecting random values based on weighting was found to be inaccurate and has been updated. Random selections with weightings will generate a different selection of results after this change. 
•  when using Get to obtain a reference to a field a default value can supplied if the field isn't found 
•  RndDateRange returns dates with time set to midnight see RndDateTimeRange 
•  Improved the profiling logic for detecting a columns data type 
•  Parameters on Transformations and Conditions now accept column references 
•  Skipping a row now deletes it from the output 
•  Column name references available for more functions, updated list in iData Reference Manual 
•  SetTimeZoneOffset replaced with SetUTCTimeZoneOffset and SetLocalTimeZoneOffset 
•  TestDataSet values set with NextXXX or ForEachXXX are now set before all other transforms take place, rather than after, so their values can be used by other transforms 
•  AutoFixAddress verification now maps addresses containing a Company Name, Property Name, or Locality, to the street/town columns 
•  AutoFixAddress verification only replaces responses with a return value of 100 and 400 by default, WithAllowedFlags() can be used to set a custom list of allowed flags. More information on flags can be found in the docs. 
•  Removed Binary Transforms as they were not useful in their current form 
•  Amended ReplaceNearest to support default value 
•  Added RntIntBetween which generates values including the maximum value 
•  Generated credit cards now give an invalid checksum by default. Added EnableValidCheckDigit() to the tag settings toa allow valid checksums to be generated. 
• ☑️ Fixed
•  Lookups from csv files will use column types defined using SetColumnToXXX 
•  Fixed infinite data generation when no row limit or foreach 
•  Report viewer now correctly reports invalid filenames instead of reporting missing files as invalid encryption keys 
•  Report viewer now correctly handles splitting of files correctly 
•  FormatText now works on fields that contain a NULL value 
•  Fixed error when using SetPatternLiteral 
•  Fixed 'UpdateColumnDefinition' which now correctly increases column sizes when ingesting csv data 
•  Miscapitalised column names used in SetColumnMapping no longer throw an error with counter group '???' 
•  AddressValidationAPI no longer requires external dependencies to run 
•  AddressValidationAPI will now create missing cache 
•  Stopped Abs transform displaying warning for positive numbers 
•  Fixed an issue when generating data that would cause the primary keys to be set to null 
•  Fixed NULL value handling when used with references to other fields 
•  #225 Fixed an issue when generating data that would cause the primary keys to be set to null 
•  CSV entries longer than 8000 characters will no longer throw an error 

Version - 1.8.0

• ❇️ New
•  Reports are compressed and encrypted to a single .idr report file by default 
•  Added SetVirtualFolder to filemanager to direct all folder contents to a single report file 
•  Encryption keys can be created and set through a encryption key pass phrase in the user interface 
•  Installer now deploys Report Viewer to view .idr files 
• 🔀 Changed
•  Profilers are now listed alphabetically 
• ☑️ Fixed
•  Fixed issue when using multiple lookups in one script 
•  Fixed profile reports with group results 
•  Fixed opening reports on a network drive accessed via a UNC path 
•  Fixed AutoFix address output miscapitalising words 
•  Fixed error when address config not set 
•  Fixed regex match error when the db value is null 

Version - 1.7.2

• ❇️ New
•  Added RestClient to allow for iData scripts to interact with Rest endpoints 
•  Added TextFileReader to allow for any text based file to be open and be used in a datastream 
•  Added TextFileWriter allowing transform engine to write to text files 
•  Added RestartLimit and RestartDisabled for Next lookup transforms 
•  Added SetTwoDigitYearMax to set the default century for two digit year date formats 
•  Added SetColumnToGuid to database readers to ensure unique identifiers are compared correctly 
•  Allow csv writers to append or overwrite existing files 
• 🔀 Changed
•  Updated ETL wizard to EnableIdentityInsert when target has identity column. 
•  MakeRndAddressCols() can use existing country or region or filter by country and region with UseCurrentCountry UseCurrentCountryAndRegion options 
•  Removed VaryBy() 
•  Added foreign key references to wizard script 
•  SetColumnToDateTime now also takes optional DateFormat to override the default 
•  CreateTableIfMissing now uses the input column name not a normalised version 
•  AddRecord now uses Upsert when there are no comparisons 
•  Added column mapping to the source stream, a compare is no longer necessary to map columns, compare mapping is being deprecated 
•  AutoFix now capitalises location names, and places houses in address line 1 
•  Dates are now formatted to the local time before being written to reports 
• ☑️ Fixed
•  Fixed data generation resetting identity columns 
•  RepeatRnd() changed to inclusive range to match Repeat() and missing ref bug fixed 
•  Allow updates to unique keys when source or destination have no rows 
•  Ensure error messages are reported for all row states 
•  Fixed error when uploading from excel sheets with numbers for sheet names 
•  Fixed TrackRowUpdates when used with existing data 
•  Fixed Importing csv file with only headers 
•  Fixed wild cards not applying tags to each column 
•  Fixed cardinality max value estimator not tracking past the set limit 

Version - 1.7.1

• ❇️ New
•  DbWriter and SQLWriter now supports creating and updating schema using CreateTableIfMissing, CreateColumnIfMissing and UpdateColumnDefinition. 
•  Added ExcelWriter allowing transform engine to write to excel sheets 
•  Added support for password protected excel files 
•  Added ExtractRegex() transform which allows text matching a given regex pattern to be extracted and optionally replaced with other text 
•  Added ToCol() which allows text extracted from ExtractRegex() to be placed in another column 
•  Added Company, Email, and Phone number to random address generation 
•  Added SetPatternLiteral() to toggle between pattern generation and literal characters in RndPatterns() 
•  Added Single, Double, and Triple word occurance count to the profiling report 
•  Added AddressLine1, AddressLine2, and AddressLine3 for generic columns 
•  Added SetColumnMappingFrom() to allow the use of lookup tables to map compared columns 
•  Added EnableAddressFixLookupService to enable the use of AFD instead of LibPostal to clean addresses, alongside EnableAddressFixMachineLearning, default option is set in AppSettings.ini 
•  Added abort to transforms when too many writes fail in a row can limit can be set with SetWriterMaxWriteFail 
•  MakeCol now accepts Wildcards to select multiple columns at once 
•  MakeCol now accepts multiple columns to be passed at once 
•  Added ExcelReader allowing transform engine to read from excel sheets  
•  ExcelReader stops reading when blank rows found 
•  New Licencing system. Contact Intelligent Delievery Solutions for support. 
•  TrackRowUpdates() now writes comments to SQL, CSV and Excel files 
•  iData Address Validation API is now packaged with the iData install 
•  Added date and time to console logs 
•  Added new conditions to match profiler results. E.G. HasPunctuation 
• 🔀 Changed
•  Lookup matches not support patterns 
•  Merged CardPrefix and CardPrefixRange for card definition in AppSettings 
•  Cards higher than 19 digits can now be validated 
•  AutoFixAddressFields now works on tables with joined number and street fields 
•  Reports no longer show [no value] when one of the rows is entirely missing. e.g. No Src/Dest or Add and Delete 
•  Column values can now be used as parameters in conditions such as Contains 
•  You can no longer edit unique key columns while comparing unless the destination table is empty 
•  Limited the UI log output to 1000 lines 
•  AutoFixAddressFields and VerifyAddressFields now work with group tags 
•  IsValid and NotValid has been added to functions VerifyEmailAddress, VerifyAddressFields, AutoFixAddressFields, CheckForeignKeys, and CheckUniqueKeys 
•  Generate more diverse set of company names 
•  By default tables stop at the first empty row, and the first empty header if headers are set 
• ☑️ Fixed
•  Prevent error when uniqueidentifier is table key and also using Rnd transforms 
•  Fixed system buffer version mismatch 
•  Fixed spelling on some Rnd... transforms from Rng... 
•  Fixed filter buttons sometimes not showing up on ETL reports 
•  Set RndPatterns() to accept punctuation. 
•  Report links with spaces now work 
•  Csv files with trialing spaces in headers now work 
•  Allow duplicate values in data for In and Replace operations 
•  Fixed some formatting issues in the documentation 
•  Data writes take column mappings into account and shows correct mapping in headers in report 
•  Fixed showing mapped columns in report when writing to empty table 
•  Fixed table name validation issue 
•  Speacial characters in excel sheet names now upload okay 
•  Address generation email column not defined and street address default seed 

Version - 1.7.0

• ❇️ New
•  More detailed stack trace for errors 
•  Added NextGuid(), RndGuid() and EachGuid() transforms 
•  Added OnNthRow() transform 
•  Added ConvertToBigInt() which converts the value to a 64 bit integer 
•  More detailed stack trace for errors 
•  Added a UniquePrefixes profiler 
•  Added RndWords() transform which generates a random Lorum Ipsum paragraph 
•  Added ability for tags to be grouped together 
•  Added enhanced Mask() which allows full or partial masking of text with a given mask character 
•  Added enhanced MaskExcept() to mask all except a given range 
•  Added Repeat/RepeatRnd modifier(s) to Each and Next Transformations to repeat the value the requested number of times 
•  Added MakeKeys and ForEachRowFrom/NextRowFrom to setup keys values from a parent table 
•  Added default value option to conversion transformations 
•  Added the ability to set the seed for randomly generated data to an alternative name instead of the column name with UseSeedName() 
•  Added AllUppercase, AllLowercase and MixedCase profilers 
•  Added Flush() to reports to write any buffered output 
•  Added ability to set RequiredRows for a TestSet 
•  Added RepeatRnd() function which allows Each and Next transforms to be randomly reset 
•  Each functions are now known as ForEach. e.g. EachText is now ForEachText 
•  Added RestartRnd() which restarts a ForEach sequence after a random amount of iterations between the given minValue and maxValue 
•  Added SetBinary(), RndBinary(), NextBinary(), EachBinary() and EachFrom versions of these 
•  Added RowLimit() to Each data items to allow for generation of data for smaller sub sets 
•  Added SetDifferenceFilename() option to comparison options 
•  All transforms can now be contained within a TestDataSet() 
•  All TestDataSet() calls must now have an accompanying EndTestDataSet() 
•  Added GetCardType() to check a card number for a matching card type, returns the found type as a string 
•  Added MatchKeysAlphbetically() method to comparison settings to allow for keys to be compared alphanumerically 
•  Added the ability to enable soundex for similar record checker via similar record tags 
•  Added the ability to ignore words for similar record checker via similar record tags 
•  Added the a sort by method the similar record configuration object 
•  Added BlankCol() to blank selected columns 
•  Added UniquePatternLengthIndependent profiler 
•  Added MakeRndCreditCardCols() to generate credit card data on tagged columns. Additionally, SetValidCardTypes() can be used to select card types to generate. 
• 🔀 Changed
•  Temp data is now written to local app data 
•  By default both MakeRndAddressCols() and MakeRndPersonCols() use predefined seeds. This can be changed with the new UseDefaultSeeds()/UseColSeeds() options 
• ☑️ Fixed
•  Prevent error when uniqueidentifier is table key and also using Rnd transforms 
•  Fixed system buffer version mismatch 
•  Fixed spelling on some Rnd... transforms from Rng... 

Version - 1.6.1

• ❇️ New
•  Added the ability to set a timeout when calling Execute() on a DBQuery 
•  Added NextIntAsc(), NextFloatAsc(), NextDecimalAsc(), NextYearAsc(), NextMonthAsc(), NextDayAsc(), NextHourAsc(), NextMinuteAsc(), NextSecondAsc() and NextMillisecondAsc() which increment a value by a given step 
•  Added the RngRegex transformation to randomly generate values that match the specified regular expression 
•  Added ability to generate synthetic data using OpenTableGenerator(). Additionally, use SetTotalRows() and UseOptimalRows to set the amount generated 
•  Added a new DataType profiler that reports back the type of each column profiled 
•  Added a new Median profiler that reports back an estimated median value when the number of unique values has been exceeded. Actual median is reported below this limit 
•  Added Host.NewDateTimeFormat() and Host.SetDefaultDateTimeFormat() which allow for parsing custom date formats 
•  Added a progress bar when the profiler is writing out html reports 
•  Added a new VaryBy transformation that can be used when obfuscating values 
•  Added OnRndRows() and OnRows() to allow for transformations to be applied on a subset of rows 
•  Added grouped profiling results to the profiling reports 
•  Added IsBlank() and NotBlank() conditional transformations 
•  Added Between() and NotBetween() conditional transformations 
•  Added option to paginate sql and csv files via a parameter on Host.WriteSqlScript() and Host.WriteCsvFile() 
•  Added the ability to edit and add new connections from the connection picker when running a wizard 
•  Added Or() conditional transformation 
•  Added SetBigInt() which sets the value to the given int as Int64 
•  Added ConvertToBigInt() which converts the value to a 64 bit integer 
• 🔀 Changed
•  MoreThan(), Equals(), AtLeast() and other comparisons will convert text values to dates before comparing them 
•  NextIntFrom(), NextDoubleFrom(),NextFloatFrom(), NextTextFrom() and NextDateFrom() have been renamed to NextInt(), NextDouble() etc when using a list of values opposed to a lookup 
•  Null values are now included in the unique values profiler results 
•  Total row count is now shown next to the Unique values on the profiling report 
•  Now when the number of unique values passes 1000(by default) an estimation of the number of unique values is provided for the profiling reports 
• ☑️ Fixed
•  Error message is displayed in reports when invalid column mappings are provided 

Version - 1.6.0

• ❇️ New
•  When updating from 1.5.* to 1.6 iData needs to be uninstalled prior to updating 
•  Added SetWriteFailedRecords() which allows failed records to be written to a separate destination 
•  Added new setting to profiler results SetMaxCharLimit() which prevents the profiling of exceedingly large text values 
•  Added NewLookup() to Host that allows the building of lookup tables from within scripts 
•  Added NextIntFrom(), NextDecimalFrom(), NextDateFrom(), NextFloatFrom() and NextTextFrom() which loop through given values replacing each value in turn 
•  Added RndTextFrom(), RndIntFrom(), RndDecimalFrom(), RndFloatFrom(), RndDateTimeFrom() and RndTextFrom() which select a random value from provided values 
•  Added RndTextPattern() which converts a pattern, e.g. 'A9A 9A9', into a random string with the same pattern e.g A7A 3R8 
•  Added SetGenerationSeedCols() to set which column values are used as the seed for generated data 
•  Added RndIntRange(),RndFloatRange(),RndDoubleRange(),RndDecimalRange() and RndDateRange() used to generate random data 
•  Added the ability to sample datastreams with the SetSampleSize() method 
•  Added the ability to generate random addresses with the MakeRndAddressCols() transformation 
•  Added Each transforms which generate data in combination with other Each transforms. Added for Text, Int, Decimal, Float and Dates. Each transforms can be used with LookupTables using Each...From 
• 🔀 Changed
•  Renamed AutoFixPhoneNumber() transformation to FormatPhoneNumber() 

Version - 1.5.1

• ☑️ Fixed
•  Fixed comparison summary not reporting differences when source rows missing from destination 
•  Fixed row count out by 1 when reading from csv files 

Version - 1.5.0

• ❇️ New
•  Added Profiling and Profiling wizard to discover data patterns and potential data quality issues 
•  Added Check for Similar Records and Similar records Wizard to search for records that could be duplicates 
•  Created data transform engine for ETL as well as comparison See NewDataTransform in docs 
•  Added transformations: ConvertToDecimal(), ConvertToText(), ConvertToDateTime(), ReformatDateTimeText() 
•  Added AutoCapitalise() transformation allowing for words to be automatically capitalised 
•  Added transformations: Trim(), TrimEnd(), TrimStart() 
•  Added the ability to tag columns as the following: TagAsPersonTitleName(), TagAsPersonFirstName(),TagAsPersonMiddleName(),TagAsPersonLastName() 
•  Intelligently clean person/name related data: AutoFixPersonFields(), DistributePersonFullName() that have been tagged as person fields 
•  Added EnableDomainCheck() on TagAsEmail to check if the email domain exists 
•  New transformations: CopyCol() and CopyColValue() 
•  New transformation: ReplaceWildcard() 
•  Added for lookup table: EnableWildcards(), EnableRegex() 
•  Added conditions: IsValid(), NotValid() to report if tagged columns such as emails are valid 
•  Added TagAsPhoneNumber() to allow tagging of phone number columns see AutoFixPhoneNumbers 
•  Added AutoFixPhoneNumbers() to convert phone numbers to standarised formatting 
•  Reports are now timestamped so results do not get lost between runs 
•  Added conditions: MatchesDateTimeText(format) and NotMatchDateTimeText(format) 
•  Added CheckForeignKey() to stream to check if all foreign keys are present 
•  Added CheckUniqueKey() to streams to check all key values are unique 
•  Added TagProfilerSetting() to allow for custom profiler settings 
•  An SQL output can be added for both CheckForeignKeys() and CheckUniqueKeys() via SetSqlResultsFilename() 
•  Common values outputted by the Profiler can be save to CSV via the WriteCommonValuesToCSV() command 
•  Added PadLeft() and PadRight() to add spaces to a value 
•  Added DifferenceFrom condition to allow for approximate string matching 
•  Added ConvertToJulianDateTime() and ConvertToUnixDateTime() 
•  The ConvertToDateTime transformation now works with unix timestamps and julian dates. 
•  Added SetLong(), SetBool(), SetDecimal(), SetGuid(), SetShort(), SetByte(), and SetDateTimeOffset() 
•  Added EnableMultipleSelections()/DisableMultipleSelections() to SetControlToListBox() for use with Wizards 
•  Added an Else statement for use with conditions 
•  The name parameter of MakeCol() can now be omitted, MakeCol() will use the column name provided in preceding IfCol() or IfSrcCol() methods 
•  Added ability to enable and disable warnings with via EnableLogWarnings(), DisableLogWarnings(), EnableAbortOnWarnings(), DisableAbortOnWarnings(), RestoreLogWarning() 
•  Added CoalesceCols() to get the first non null value from a list of columns 
•  Added Substring() transform to get part of a text value 
•  Added JoinCols() transform to combine several colums into one value 
•  Added InsertText() and InsertColText() transform to add text at the begining or middle of a value 
•  Added TagAsDomain() to allow tagging of a web domain column 
•  Added NewDBResultsWriter() to the Host object 
•  Added TrackRowUpdates() and StopTrackRowUpdates() 
•  Added SetDNSServer(), ExportDNSCache() and ImportDNSCache() 
•  Added TagAsCreditCard() and validation to check if credit card numbers are valid 
•  Added EnableSoundex() to lookup tables 
•  Added Obfuscate() transform to replace original characters with random characters 
•  Added Mask() transform to replace original characters with fixed characters 
•  Profiling Results can now be grouped by columns 
•  Lookup tables can now be filtered using FilterOn() and Using() 
• 🔀 Changed
•  Database connections and application settings are now read from %LocalAppData%/iData/Settings 
•  Data comparison engine still works but is deprecated, use NewDataTransform instead 
•  NewInputStream still works but is deprecated, use NewStream instead 

Version - 1.4.0

• ❇️ New
•  Upgrade to V8 jscript engine, for better performance, error handling and latest jscript syntax 
•  Doubled performance running scripts in gui with background thread 
•  Allow debugging scripts from VSCode. See Docs 
•  Allow scripts to be stopped from the UI by clicking the run button a second time 
•  NewDbCompare can use SetSrcData or SetDestData instead of SetSrcTable or SetDestTable. SetSrcData can be passed value from OpenCsvFile. This allows control over csv file separators and column settings 
•  NewDbCompare can use SetSrcQuery or SetDestQuery to allow comparison of joined tables or pre-processing of data using sql before iData transformations 
•  Use a lookup table from a query or csv file to replace values using ReplaceValueWithLookup 
•  The database definition of a Null value can now be accessed through Host.DBNull which has a different value from null 
•  IgnoreCase can be applied to make transformations such as Replace case insensitive and MatchCase can turn it case sensitive again 
•  Added SetInt, SetFloat, and SetDateTime to set a column to a specific value of the given type 
•  Created maths operations for Add, Subtract and Remainder 
•  SetMatchDistance added to allow nearest close match to data in lookup table. See Docs 
•  Added SetDateTimeParts to allow part of a dateTime to be changed. 
•  Added SetANSICodePage to allow encoding to be specified when reading CSV files. 
• 🔀 Changed
•  Interactive scripts must have 'Wizard' as part of their filename, for automatic optimisation 
•  ReplaceRegex ignore case parameter deprecated in favor of IgnoreCase command 
•  ReplaceValueWithLookup deprecated and renamed ReplaceLookup 
• ☑️ Fixed
•  Fixed invalid warning when comparing source query to destination csv file 
•  Fixed crash caused by duplicate database connection names 
•  Report any errors generated by wizards 
•  Prevent ConfirmRequiredVersion flagging identical build numbers as different 
•  Prevent missing dates in CSV files generating invalid SQL dateTimes 
•  Fixed cleanup of database connections after DbCompare 

Version - 1.3.6

• ❇️ New
•  Add testConnection wizard. See getting started 
•  Accept jscript regex expressions in MatchRegex and NotMatchRegex. See Api docs 
• 🔀 Changed
•  Database connections details reloaded on demand. No need to re-start gui to refresh 
•  Improved error message for script errors in comparisons 
• ☑️ Fixed
•  Update progress report when skipping rows during comparison 

Version - 1.3.5

• ❇️ New
•  Recored and report statitics for each column 
•  Accept native jscript regex expressions in MatchRegex and NotMatchRegex. See Api docs 
•  Remove duplicates rows before comparison. See RemoveSrcDuplicates in Api Docs 
• 🔀 Changed
•  Limit comparison reports of mismatched rows to avoid hiding other issues 
• ☑️ Fixed
•  Report attempts to map columns incorrectly in comparisons 
•  Prevent installer overwritting connection strings on re-install 

Version - 1.3.4

• ❇️ New
•  Allow comparison to specify IncludeColumns as well as SkipColumns. See Api docs 
•  Allow columns to be SetNull. See api docs 

Version - 1.3.3

• ❇️ New
•  All In to use lookup tables from CSV or queries. See API docs 

Version - 1.3.2

• ❇️ New
•  Additional conditions In, StartsWith etc. See API docs 

Version - 1.3.1

• 🔀 Changed
•  Improved database connection error messages 

Version - 1.3.0

• ❇️ New
•  Support for splitting and joining columns. e.g. AddColText. See Api docs 

Version - 1.2.0

• ❇️ New
•  Database comparison wizard 
•  Import CSV data for comparison 

Version - 1.1.0

• ❇️ New
•  Database comparison 
•  Read/write to Excel files