JADE  Environment Development Ideas

Ability to use a copy of a production database for test purposes while disguising sensitive data

The objective is to copy a production system for testing purposes with designated data values disguised without having to scramble the data.

Initial requirements:

1. This would assume that the database would be encrypted to avoid direct examination of the database files.

2. Properties would be "marked" in some way as part of the normal development process.

3. Some data (such as post codes) would need to be translated to specified value(s) because validation is performed against a third party system. This could be just another type of "marking" the property.

4. Some data (such as credit card numbers) need to have some (or all) of their contents masked - once again, this could be just another type of "marking" the property.

5. Any disguising of data would need to be done in a consistent manner - for example, a customer's name may be present on multiple objects having different attribute names, but it would need to be presented the same.

6. Functions like jade audit access, journal dumps, stack dumps, and debugger would need to present the disguised values.

7. Any new data (or changes to existing data) made in the test database would not be subject to these rules. The data would be presented in its original form.

  • Keith Domigan
  • May 26 2020
  • Needs review
  • Attach files
  • John Beaufoy commented
    26 May 03:49am

    Hi Keith,

    We rolled our own solution for this.
    A multi-threaded script is run as part of setting up a test base to desensitise the data.

    Probably the main point of difference on your solution is we change the db values. We didn’t want different code paths for live and test, and we didn’t want to leave any risk that the masked data could be unmasked again.

    We've otherwise catered for all your other requirements.
    Let me know if you want some further information on how it works.
    It was written to be portable, so if there was demand for it we could check if it could be made available.