MySQL

Sifflet monitors the health and quality of the data in your MySQL databases, from table updates down to individual field values and business metrics. Sifflet catalogs the tables and views of your database, so you can monitor them and see how an incident on your production data affects the downstream assets of your other integrations.

To learn what Sifflet imports from MySQL, see MySQL Collected Data.

Permissions Required

Sifflet uses a dedicated MySQL user that authenticates with a password. Sifflet only needs read access: it reads metadata about your tables, and runs read-only SELECT queries to compute monitors.

Sifflet's monitor queries run on your database and consume its resources. Each query starts with a SQL comment containing "app": "Sifflet" and the ID of the monitor, so you can identify Sifflet's queries in your database logs.

Requirements

Required Roles for Setup

• In MySQL: a user that can create users and grant privileges on the monitored database (CREATE USER and GRANT OPTION privileges, for example an administrator account).
• In Sifflet: the Admin system role to create the credential, and the Admin or System Editor system role to create the source. See Access Control.

Supported Versions and Editions

• Sifflet connects to MySQL with username and password authentication. Other authentication methods, such as client certificates or cloud IAM database authentication, are not supported.
• Managed MySQL services that expose a standard MySQL endpoint use the same setup. This includes Google Cloud SQL for MySQL, with built-in database users: see Cloud SQL for MySQL for its network setup.

Network Access

• Default: Sifflet connects to your database over the internet, so its host and port must be reachable from the public internet.
• IP allowlisting: you can restrict inbound traffic to your database (firewall rules, cloud security groups) to the IP addresses of your Sifflet instance, listed in Settings > Integration Preferences. You can also restrict the host from which the Sifflet user connects (see step 1).
• Private networks: the Sifflet Agent supports MySQL: deploy it in your private network so that your database doesn't need to be exposed to the internet.

Cloud SQL for MySQL

Sifflet connects directly to the IP address of your Cloud SQL instance: it doesn't use the Cloud SQL Auth Proxy or the Cloud SQL connectors, so the instance must not require connectors. See Connect to Cloud SQL for MySQL.

• Public IP: turn on public IP for the instance, and add the IP addresses of your Sifflet instance (Settings > Integration Preferences) to its authorized networks. Use the public IP address of the instance as the Host of the source.
• Private IP only: deploy the Sifflet Agent in a network that can reach the private IP address of the instance, and use this address as the Host of the source.
• Sifflet doesn't support client certificates: don't require trusted client certificates on the instance.

Connect Sifflet to MySQL

Step 1: Create a Database User in MySQL

1. Connect to your MySQL server as a user that has the rights listed in Required Roles for Setup.

2. Run the following script. Replace <PASSWORD> with a strong password and <DATABASE> with the name of the database that Sifflet monitors. You can choose another user name than sifflet_user: use it consistently in the whole script.

   SQL

   CREATE USER 'sifflet_user'@'%' IDENTIFIED BY '<PASSWORD>';
   
   -- Repeat for each database that Sifflet monitors (SHOW VIEW is optional, see Permissions Required)
   GRANT SELECT, SHOW VIEW ON `<DATABASE>`.* TO 'sifflet_user'@'%';

   The % host lets the user connect from any host. To restrict it, replace % with a specific IP address or host name, in every statement. If Sifflet connects directly to your database, the connections come from the IP addresses listed in Settings > Integration Preferences: create one user per IP address, with the same user name and password. If you use the Sifflet Agent, use the address of the Agent.

Cloud SQL: Create the User with SQL

Run the script above as the root user (or another user with the cloudsqlsuperuser role). Create the Sifflet user with SQL rather than in the Google Cloud console, so that you control exactly which privileges it gets.

Keep the user name and password: you need them in step 2.

Step 2: Add the MySQL Source in Sifflet

1. Create the credential: in Integrations > Credentials, click New credentials, enter a Credential name, and paste the following JSON in the Credential field, with the user name and password from step 1:

   JSON

   {
     "user": "sifflet_user",
     "password": "<PASSWORD>"
   }

2. Create the source: in Integrations > Sources, click New source, select MySQL, and fill in the form:

   Field	Description	Example
   Source name	The name of the source in Sifflet	MySQL production
   Host	The host name or IP address of your MySQL server, without protocol or port	db.example.com
   Port	The port of your MySQL server (default 3306)	3306
   Database	The name of the database that Sifflet monitors	app_db
   TLS version	The TLS protocol version that Sifflet allows when connecting to your server: TLSv1.2 (default) or TLSv1.3	TLSv1.2
   Use Sifflet Agent	Only if you use the Sifflet Agent: check it so that the Agent connects to the database. This option is visible when the Agent is turned on for your Sifflet instance.	-
   Credentials	The credential created above	mysql-production

   Use the same host as the other tools that connect to this database (BI tools, dbt, pipelines): Sifflet matches the host, port, and database to link their assets in lineage.

3. Scope: Sifflet catalogs all the tables and views of the database entered in the form. Create one source per database. For more details on refresh frequency and scheduling, see Integrations Management.

4. Test and save: click Test to check that Sifflet can connect to the database, then click Save & Run. Sifflet starts the first metadata refresh: you can follow its status and logs in the source details (see Inspecting Source Details and Resolving Failures). Your tables and views appear in the Data Catalog once this first run completes.