Building integrations with Monosync
What changed
Every content update is versioned. Select a version to compare it with the one before.
Version 7 → 8
UpdateNew module on integration jobs in depth so the course covers the MCIE exam: platform sources, schema reload, variables in each syntax, mapping errors and row order, removal actions, links, the object API, export and import, timeouts, stopping and recovering runs. Corrected: an action that throws deactivates the integration. Diagrams and first-check lists were added to the new lessons.
3 lessons updatedduration
Content update. Enrolled learners were notified.
Duration changed from 22 to 25 minutes.
An integration that works in a demo can still behave badly with real data. The causes are nearly always a handful of rules about sources, schemas and mapping rows. This lesson walks through them in the order a run meets them.
### Reading the Monosync object store
A source of type `Integration Platform` reads Monosync's own objects. Its setting `Use only updated items` is what makes an `Incremental Sync` small: without it an incremental run takes every object and lasts as long as a full one. Do not look for the answer in `Priority`. That field is stored but the run code does not read it.
### Reload resets your flags
`Reload` on a schema that already exists replaces every non-custom property with the provider's current list. Properties you created yourself stay, but `Key`, `Identity` and the other flags on provider properties go back to the provider's values. A Link run matches on the `Key` property of the target schema and stores the `Identity` value, so after a careless reload existing users are no longer found and a Create action meets accounts that already exist. Check the flags after `Reload` and before `Save`.
### Secrets belong in variables
A source setting with the lock icon is only masked on screen. The source's settings are stored as plain JSON, and every save of a source writes a commit row. A variable marked as encrypted is the real encrypted store. Put the password into such a variable and reference it in the source setting as `{{Variables.AdPassword}}`. The value is replaced at run time and when you test the connection.
### Three places, three spellings
| Where you write it | How a variable is written | Example |
| --- | --- | --- |
| Source settings and action parameters | Double braces | `{{Variables.MailDomain}}` |
| `Formatted` modifier on a mapping row | Single braces, and `{0}` for the incoming value | `{0}@{Variables.MailDomain}` |
| `Code` modifier | A C# name | `Variables.MailDomain` |
With the `Formatted` template above, the value `jdoe` and the variable `MailDomain` = `example.com` give `[email protected]`. A variable written in the wrong form is not replaced, and the row fails with a mapping error.
### What a failing row does
A mapping row that throws is logged for that record and the record is skipped. The run goes on. It ends in error only when every record fails. A `Code` row has to check for empty values itself. You find the failed records under `Mapping error(s) found!` on the run's actions view, and with `Include Action(s) failed on last run(s)` on, which is the default, later incremental runs pick them up again once the code is fixed.
### Order between rows
Rows are evaluated in parallel, with one exception: a `Code` row that assigns `Data.X = ...` runs serially before all the others. If one row builds the e-mail address from a user name that another row computes, the first row must assign the computed value through `Data.X`. `One Time` does not help here. It behaves like `One Way`.
::diagram scd-mapping-order
### Why a Link run sees a change that is not there
A Link run compares scalar values by their text form, and the comparison is case-sensitive. A boolean produced by a mapping is written as `True`, which never equals a target value of `true`, so the same users are marked for update on every run. Open the `Values` modal of such an object: `Before` and `Now` differ only in case or type. The fix is to normalize the value in the mapping, for example to lower-case text.
### Functions are compiled at every run
Every function is compiled again when a run starts. `Remove` on a function checks mapping code only. A function that is still called from an interceptor, a filter or another function can be removed, and from then on each caller fails to compile and every run of the affected integrations ends in error. Use `Show References`, and search interceptor and filter code for the name, before you remove a function.
> Tip: When a run surprises you, check in this order: the `Key` and `Identity` flags after a `Reload`, the spelling of each variable for the place it is written in, and `Mapping error(s) found!` on the run's actions view.
After mapping, the records reach the actions. This is where an integration writes to a real system, so the rules here matter most.
### Where to filter
| Place | Sees | Use it for |
| --- | --- | --- |
| Rule builder on the integration's `Sources` tab | Source records | Selecting rows on the source side |
| `After Source Fetch` interceptor | Raw fetched fields of the chosen sources | Dropping or changing records with code before mapping |
| `After Mapping` interceptor | Mapped records | Adjusting the mapped list before the actions |
| Filter on an action | The mapped, target-side values | Deciding which records this action receives |
A field that is not mapped anywhere can only be tested in the first two places. An exception in an interceptor ends the run in error, so prefer the plain source filter when it is enough.
::diagram scd-filter-places
### Removal needs `None` and a filter
An action receives the records that match its `Projection`. `Create` and `Update` follow the object's link. A removal-type action, such as disabling the accounts of leavers, is the odd one: in sync runs the `Remove` projection selects no record, and the action log says `No need to take action. There is no data for this projection.` Configure the action with projection `None` and a filter that selects the leavers. Because action filters read mapped values, the leaver indicator must be a mapped property.
### A failed action and a thrown exception are different
| What happens in an action | Effect on the run |
| --- | --- |
| Some items end with Warning or Error | The action log shows it, and the next action still runs |
| The action throws, for example an invalid filter or a provider crash | Later actions do not run, the run is set to Error and the integration is deactivated |
In both cases the work of earlier actions stays applied. Nothing is rolled back. After an exception you fix the cause and activate the integration again.
### Passing record data to a script
The PowerShell `Execute Script` action can receive record values in two ways. A `{{Data.displayName}}` placeholder is pasted into the script text, so a value with a quote or a semicolon changes the code that runs with the rights of the configured account. A named entry under `Parameters` is passed as a value:
```
param($DisplayName, $Sam)
Set-ADUser -Identity $Sam -DisplayName $DisplayName
```
Use `Parameters` for anything that comes from record data.
### Links decide Create or Update
A Link run matches each object to a target row on the target schema's `Key` property and stores the row's `Identity` value in the link. Choose a `Key` that is stable and unique in both systems, such as an employee number. Display names change and repeat.
A link is found by object, source and provider, not by integration. If integrations A and B write to the same source, a link made by A also makes B treat the object as linked. Clearing the links of B leaves those of A in place, and B keeps sending the object to its Update action.
### Writing objects through the REST API
```
PATCH /api/v1/objects/employee/1042
Api-Key: <key>
{ "department": "Sales" }
```
| Method | Properties missing from the body |
| --- | --- |
| `POST`, `PUT` | Their values are removed |
| `PATCH` | Their values are kept |
Every API write records an object change, even when no value changed, so the object enters the next incremental run. Send only real changes.
### Export and import between environments
`Settings` exports chosen integrations, sources, object types, variables and functions to one JSON file and imports it elsewhere, item by item. An incoming item is matched by ID, then by name, and a match is replaced only with `Overwrite all objects`. Before you import into production with that option, remember three things: selected sources bring their test hosts and passwords with them, so deselect them; an imported or overwritten integration is left inactive, so activate it afterwards; and object records are not part of the export.
A run is the moment an integration actually touches a target system. Most surprises in operation are about three questions: when a run starts, why one did not start, and what is left behind when one stops halfway. This lesson answers them in that order.
### Four ways a run starts
| Start | When | Remember |
| --- | --- | --- |
| Schedule | A cron time on the `Triggers` tab, evaluated in UTC | One run mode per schedule |
| Unscheduled loop | The integration is active and has no enabled schedule | Runs back to back |
| `Run Once` | Only on an inactive integration | Commits real writes |
| `Calculate` | Any time, with a chosen mode | A dry run that never calls target actions |
The unscheduled loop surprises people who activate an integration before adding its schedule. The integration does not wait. It runs immediately and again after each run, as a Full Sync when no successful unscheduled Full Sync started within the last day, otherwise as an Incremental Sync. Add the schedule first, then activate.
### Missed times are not caught up
The next schedule time is the next cron occurrence after the later of two moments: the last job start of that schedule and the activation. An integration that was deactivated over a weekend does not run the two times it missed when you activate it on Monday. Ending maintenance mode behaves the same way. If the missed work is needed, start it deliberately.
### One run per integration
Monosync never runs two jobs of one integration at once. A run that would overlap is skipped with `This integration is running in another task.` Its row on `Integration Jobs` stays idle and is later closed as Aborted. Several Aborted rows with empty `Start / End` are therefore skips, not failures. A common cause is a schedule that fires more often than a run lasts.
### The timeout
| Setting | Value | Meaning |
| --- | --- | --- |
| `Timeout (in seconds)` on the integration | Above 0 | The run is cancelled after that many seconds |
| `Timeout (in seconds)` on the integration | 0 or empty | The service default applies |
| `MONOSYNC_SERVICE_INTEGRATION_TIMEOUT_SECONDS` on the service nodes | Default `14400` | Four hours for every integration without its own timeout |
A run that reaches the limit ends Aborted. A full sync that needs five hours on a default installation is aborted every time until you give the integration its own value, for example `21600` for six hours. `Cron Buffer (in minutes)` and `Priority` are stored but not read, so they do not help.
### Stopping a run
`Integration Jobs` is read-only and has only a `Refresh` button. To stop a running job, deactivate the integration and answer the question with `Yes, stop also running tasks`.
### After an aborted run
Target writes are saved action by action, and an object change is recorded as processed only when its processing succeeded. Two consequences follow from an abort in the middle of the actions:
- The writes that were already made stay on the target.
- The next incremental run takes the same object changes again.
The data date of the integration advances only after a successful Sync, so nothing is lost. But the target actions must tolerate a repeat: an update that is applied twice has to give the same result as one applied once.
::diagram scd-run-after-abort
### What to check first
- Runs back to back: is the integration active without an enabled schedule?
- Aborted rows with empty `Start / End`: these are skips, so compare the schedule interval with how long a run lasts.
- A long Full Sync that always ends Aborted: give the integration its own `Timeout (in seconds)`.