
Required pre work
In order to connect Nekt to a database, you have to do some pre work to ensure access is granted in a secure way. So, make sure to check this guide before proceeding.1. Add account access
- Once you have done the pre work defined in section 0, you can inform your database accesses. In the Sources tab, click on the “Add source” button located on the top right of your screen. Then, select the Postgres option from the list of connectors.
- Click Next and you’ll be prompted to add your database access.
You must ensure your database accepts incoming connections from Nekt, and the database user has
SELECT permissions in order to fetch data.- Click Next.
Advanced Configuration: Filtering Tables
Advanced Configuration: Filtering Tables
If you have a large database with many tables, you can use the Using wildcards:This would match:
filter_tables configuration option to speed up discovery and limit which tables are available for sync. This is particularly useful for databases with hundreds or thousands of tables.Table Filtering Format:Tables can be specified as a comma-separated list with the following patterns:schema.table- Match a specific table in a specific schematable- Match a table with this name in any schema- Wildcard patterns using
*(matches any sequence) and?(matches single character)
user_accounts,user_profiles, etc. (in any schema)dbo.order_items,dbo.order_history, etc. (in dbo schema only)temp_2023,temp_test, etc. (exactly 4 characters aftertemp_)
filter_tables is not specified, all tables in the selected schemas will be available.2. Select streams
Choose which data streams you want to sync - you can select all streams or pick specific ones that matter most to you. We recommend selecting only the tables that are useful for you from you. You can add new tables at anytime after the source is created.Tip: The stream can be found more easily by typing its name.Select the streams and click Next.
3. Configure data streams
Customize how you want your data to appear in your catalog. Select a name for each table (which will contain the fetched data) and the type of sync.- Table name: we suggest a name, but feel free to customize it. You have the option to add a prefix and make this process faster!
-
Sync Type: you can choose between INCREMENTAL and FULL_TABLE.
- Incremental: every time the extraction happens, we’ll get only the new data - which is good if, for example, you want to keep every record ever fetched.
- Full table: every time the extraction happens, we’ll get the current state of the data - which is good if, for example, you don’t want to have deleted data in your catalog.