Data Cleaner

Overview

The Data Cleaner module ensures a detailed and extensive search for Personal Identifiable Information (PII) through all pages in Confluence and all tickets in Jira. In addition, it has many built-in tools, which are facilitating and automating the work with sensitive data at all stages: tracking, sending comments and notifications, reporting, and ample opportunities for customization.

The main provided features are:

  • pattern-based search (most popular and widely used built-in patterns and custom search via regular expressions)

  • search of a specific user's personal data

  • automated comments and notifications for other team members

  • rules triggered by event, allowing to track all changes (creation of ticket/page/blog post, etc.) in real time

Data Cleaner Dashboard

The starting point of working with the Data Cleaner is the Data Cleaner Dashboard. 

Find the Apps tab on the top of the start page, click on it and choose the Data Protection and Security Toolkit in the dropdown menu. After that, you’ll see the Data Protection and Security Toolkit Home page, where you’ll find the Search for personal data section with the Data Cleaner button. Click on it and view the Data Cleaner Dashboard.

Data Cleaner Dashboard

The Data Cleaner Dashboard provides an overview of all created templates for search and data processing (short description, current status) and allows you to:

  1. Quickly enable or disable templates

  2. Start or stop the search

  3. Navigate to the

Actions' menu to edit, clone, or delete the template, to view its status and history, or to clean history

Also, you see two buttons – the Create custom template button for creating a template with parameters set by yourself and the Create search user template button for creating a template with predefined parameters. Let’s find out how to create a new template.

Templates creation

Custom template

Click on the Create custom template button to create your own template. After that, you will see a page with two sections and fields that you should fill in:

  • General configuration

  • Fields processing

General configuration

General configuration

Let’s take a look at the General configuration parameters in detail:

Parameter Name

Default Value

Description

Parameter Name

Default Value

Description

Template name

Test template

Enter any name that will separate the template from others. After that, you will see the results of automatic check up, showing if the name is vacant or not.

Description

Empty

You can add a short description of your template or leave it empty.

Scope

Empty

Define the scope of tickets in Jira or pages in Confluence to be involved in the search process. Use the Jira Query Language (JQL) for the search in Jira and Confluence Query Language (CQL) for the search in Confluence.

Find some options under the Scope field which allows you to set the scope quickly – All tickets, Created last 30 days, Resolved more than 1 year ago for Jira and All pages and blog posts for Confluence. Just click on the appropriate option, and this scope will be set for the template.

Owner

Current user name

Define who manages the template and can make changes to it. Select the user from the dropdown menu who is supposed to be an owner of the template.

Actor

Current user name

Define the user in whose name the template actions will be performed (e.g., comment addition). Select the user from the dropdown menu who is supposed to be an actor.

Trigger

Manually

Select the template execution type – Manually or By content event.

Manually means you start the search by your own using the Start button on the Dashboard.

By content event means the search starts when some event happens. Choose events in the Trigger events field, which appears when the By content event type is selected:

In Jira

  • On issue created

  • On issue updated

  • On comment created

  • On comment updated

In Confluence

  • On page/blogpost created

  • On page/blogpost updated

  • On comment created

  • On comment updated

Note: it is not recommended to select all options at the same time.

Note: if you create several triggered by content event templates, only one template can be enabled.

Notify in case of error

None

This feature is only for the case of a manual template start.

It allows you to set up notifications about any error during the search. To do this, select the Template owner in the dropdown menu, and notifications will be sent to the template owner. Leave None if notifications are not needed.

Notify in case of successful execution

None

This feature is only for the case of a manual template start.

It allows you to set up notifications in case of a successful template execution. Select the Template owner in the dropdown menu, and notifications will be sent to the template owner. Leave None if notifications are not needed.

After all the fields are filled in, click the Save and Continue button and navigate to the Fields processing tab.

Fields processing

In the Fields processing section, you set what fields the search must go through, create rules and add actions that will be applied to the rules.

 

Fields

Here is an overview of the fields settings:

Parameter Name

Default Value

Description

Parameter Name

Default Value

Description

Fields

Empty

Here, you can set which fields the search should go through. It can be:

In Jira

  • Summary

  • Description

  • Comment

  • History

  • Creator

  • Assignee

  • Reporter

  • Attachment name

In Confluence

  • Title

  • Body

  • Comment

  • Attachment name

Fields format (in Confluence only)

Use rendered fields

Choose one of the options by radio button – Use rendered fields or Use raw fields format.

Use rendered fields means showing you data that the server stores in the database in a styled HTML table.

Use raw fields format means showing you data in a simple format, which the server stores in database. This option excludes macros.

 

After you set the fields parameters, navigate to the rules settings below on the page.

Rules

In this area, you can define what exactly you are going to search for. Using rules settings, you can:

  • create a new rule

  • add rules from the library

  • check rules on the issue for Jira, or check rules on the page for Confluence

Let’s take a look at it in detail.

  • New rule creation: In order to define your own custom rule, click on the Create new rule button and view the Create rule window.

 

Here is an overview of the rule parameters which you can set:

Name

Default Value

Description

Name

Default Value

Description

Rule name

Empty

Enter any name that will separate the rule from others.

Search type

Plain text

The object for search can be defined as plain text, regex, and user – just pick one of them.

What to search

Empty

Enter the text, regex, or select a user from the dropdown menu which you want to find.

Fill in all the fields and click the Save button to save your settings. After that, your rule will be displayed on the page.

 

  • Adding rules from the library: In order to use built-in rules, click on the Add rule from library button and pick one or multiple built-in rules, then click on Add selected rules

You can tick the box Select all and create a template with all built-in patterns, or narrow down the scope by further filtering the data.

Built-in patterns can be filtered by a particular country (for example, Germany, Austria, Italy) or a personal data type (for example, phone and credit card numbers), or both. To use the filter, start typing the name of the country or the personal data type, and the system automatically will show existing built-in patterns.

A Jira Administrator is able to edit, disable and delete the rules – just click on the settings button for the required rule.

Note: After the creation, the new rule has an enabled status by default. In order to disable it, click on the settings button and choose Disable.

 

  • Checking rules on the issue (Jira)/page (Confluence): Using this option, you can preview the content that matches the rule in the exact ticket of Jira or the exact page of Confluence. To do that, select your fields for searching and create at least one rule. After that, you can:

    • in Jira: click on the Check rules on the issue button and enter the ticket key in the Check issue matches window

    • in Confluence: click on the Check rules on the page button and enter the page title in the Check page matches window

If there is any data that matches the rule in this ticket/page, you’ll see the table with all found items right in the Check issue/page matches window:

 

After you set the rules, navigate to the actions settings at the bottom of the page.

Actions

The app allows you to leave standard comments on all vulnerable tickets/pages and to create and send automated notifications to employees responsible for these tickets/pages. If the personal data are not important for the company or were added by mistake, they can be found and replaced by any text, for example, XXX or ### combination. Click on the Add action button and fill in the fields of the Issue Action window:

Let’s take a look at the actions parameters in detail:

Parameter Name

Default Value

Description

Parameter Name

Default Value

Description

Action name

Empty

Enter the action name in this field.

Worked with rules

Empty

Pick the rule from the dropdown menu for which the action should be performed.

Use any rules

Disable

Enable the checkbox if you want to apply the action for all rules.

Action type

Empty

Choose one of the action’s types from the dropdown menu:

Add comment: when you choose this action, you can set the comment text in a Comment field and this text will be displayed for all affected objects.

Send notification email: when you pick this action, just enter addresses of the recipients, subject, and text of your message in appropriate fields, and your notification will be sent to required recipients.

Restrict VIEW and EDIT for content (for Confluence only): this action allows you to restrict viewing and editing of affected pages, so all the found pages will be hidden from all users.

Send notification email to template owner: when you select this action, just enter the subject and text of your message in the appropriate fields. The message will be sent to the template owner’s address automatically.

Add label: when you pick this action, you can set the label in the Label field to all affected objects.

Webhook: this action allows you to create tickets in Jira for the affected content automatically by sending a webhook callback to a predefined URL with custom headers and body. Just set the API URL, name of the ticket in the Headers field and its short description in the Body field, and the ticket will be created with this information.

Show warning on the page view: when you choose this action, you can set a popup window that will be displayed on the affected pages or tickets. To do this, enter the title in the Title field and text of the notification in the Body field.

Set content property: this action allows you to add some predefined content entity property to affected pages or tickets. For this action is required to fill-in 2 fields: Property Key is the key of the issue property (the maximum length is 255 characters, it should be one word, no spaces, no special chars) and Property Value
the value of the request body must be a valid, non-empty JSON blob (the maximum length is 32768 characters).

Replace with text: this action type could be performed for text fields only (like Summary, Description, and Comments). When you choose this option, enter the text you want to replace the found text with.

Click the Save button in the Issue Action window after setting the actions.

Now, your template is completed.

Predefined search user template

Click on the Create search user template button to create a predefined template. Firstly, you will see a window where you should select the user whose information you want to find:

  • in Jira: select username from the dropdown menu, and click on the Create button

  • in Confluence: start typing a username in the field, find the user in the dropdown menu, and click on the Create button

After that, you will see the page with two sections:

  • General configuration

  • Fields processing

As the template is predefined, all its parameters are already set up, and you can directly start using it.

General configuration

Let’s take a look at the General configuration parameters in detail:

Parameter Name

Default Value

Description

Parameter Name

Default Value

Description

Template name

Search user data {user name}

The name is predefined and will be displayed at your Dashboard.

Description

Empty

You can add a short description of your template or leave it empty.

Scope

  • In Jira: project is not EMPTY

  • In Confluence: (type = page) OR (type = blogpost)

Here you can define the scope of tickets in Jira or pages in Confluence to be involved in the search process. Use the Jira Query Language (JQL) for the search in Jira and Confluence Query Language (CQL) for the search in Confluence.

Predefined JQL and CQL requests involve all the tickets, pages, and blog posts in the instance.

Owner

Chosen user name

Define who manages the template and can make changes to it. Select the user from the dropdown menu who is supposed to be an owner of the template.

Actor

Chosen user name

Define the user in whose name the template actions will be performed (e.g., comment addition). Select the user from the dropdown menu who is supposed to be an actor.

Trigger

Manually

The template execution type Manually means you start the search by your own using the Start button on the Dashboard.

You can change it to By content event type, which means the search starts when some event happens. Choose events in the Trigger events field which appears when the By content event type is selected:

In Jira

  • On issue created

  • On issue updated

  • On comment created

  • On comment updated

In Confluence

  • On page/blogpost created

  • On page/blogpost updated

  • On comment created

  • On comment updated

  • Note: it is not recommended to select all options at the same time.

Notify in case of error

None

This feature is only for the case of a manual template start.

It allows you to set up notifications about any error during the search. To do that, select the Template owner in the dropdown menu, and notifications will be sent to the template owner. Leave None if notifications are not needed.

Notify in case of successful execution

None

This feature is only for the case of manual template start.

It allows you to set up notifications in case of a successful template execution. Select the Template owner in the dropdown menu, and notifications will be sent to the template owner. Leave None if notifications are not needed.

 

Fields processing

In the Fields processing section, there are settings of the fields the search must go through and settings of the search rules and actions.

Fields

Here is an overview of the field settings:

Parameter Name

Default Value

Description

Parameter Name

Default Value

Description

Fields

In Jira

  • Summary

  • Description

  • Comment

  • History

  • Creator

  • Assignee

  • Reporter

In Confluence

  • Title

  • Body

  • Comment

Here, you can set which fields the search should go through.

Fields format (in Confluence only)

Use rendered fields

The format Use rendered fields means showing you data that the server stores in the database in a styled HTML table.

You can change it to Use raw fields format, which means showing you data in a simple format the server stores in database. This option excludes macros.

After you set the fields parameters, navigate to the rules settings at the bottom of the page.

Rules

In this area, you can define what exactly you are going to search for. Rules Search user key, Search user email, Search user name, and Search user data {user name} are already added and enabled.

Actions

There are no actions in the template by default, but you can add them if necessary.

Click the Save button in the top right corner of the page to save your changes of the template.

Search

On the Dashboard, you see the created template.

Click on the Start button in Actions column to start the search for personal user data. It can take some time depending on the amount of content in your Jira or Confluence.

While the search performs, you can see the information panel at the top of the Dashboard where it is indicated which template is running, by whom it is run, and its current status:

 

When the search is done, you see a check mark in the History column. After every execution, you will see a new check mark next to the previous one. Click on it, and you will see search results for the respective execution on the History page:

 

 

In Jira, on this page, you see project names where affected content was found (Project column), keys of the affected tickets (Key column) and the names of the ticket (Summary column), part of the ticket, where the personal data were found (Field column), names of the performed rules (Rule column) and names of the related actions (Action column), was the action applied or not (Status of action column), and the Check case feature (Inspect column), allowing you to form an ignore list. See more information about the ignore list below.

In Confluence, on this page, you see space names where affected content was found (Space column), IDs (ID column) and names (Title column) of the affected pages or blog posts, fields where the data was found (Field column), names of the performed rules (Rule column) and names of the related actions (Action column), was the action applied or not (Status of action column), and the Check case feature (Inspect column), allowing you to form an ignore list. See more information about the ignore list below.

You can filter the results by project/space, object, status, and rule, and export the results in CSV or JSON format.

 

Ignore list

Let’s take a look at the Check case feature in detail.

Click on it, and view the Case window, where the information about the exact case is displayed:

In this window, you can see where the affected content was found, what rule was performed for this, and the content itself. If it’s necessary to skip certain content in subsequent searches, choose the Ignore certain content, field and rule option in the Take action dropdown menu and click Save. After that, when you start the search, this match in the certain field will be ignored.

 

To skip some text, that you set in the search rules, pick the Ignore specific text option and enter the text in the Text to ignore field. After that, when you start the search, this match will be ignored in all the fields.

If you set content to ignore, you see which specific texts or rules are ignored in the Fields processing section of the template settings:

 

You can delete any elements from the ignore list when necessary.

 

Now, you are ready to manage the sensitive data with Data Cleaner in your instances.