Import Task | Memority Documentation

Definition

An Import Task allows either an end user or the system (in the case of a scheduled import) to launch a data import using a CSV file or a REST API.

You can choose to:

Start the import immediately
Schedule it to run once at a specific date and time

Usage

An Import Task is either:

Private – An end user launches it manually.
Public – A scheduled job runs it at regular intervals.

Launch an Import - Private Import

When a user launches a private import, only that user can access the task. Users with administrative rights can also view and manage all private imports.

	Steps	Preview
1	In the User Portal, go to Administration > Import Tasks, then click Run Task.
2	Select the import task you want to run. If the import type is CSV, choose one of the following options: Upload a CSV file from your device Use a file available on the Memority Exchange server If the import type is LDAP, REST API, or another supported source, skip this step, you don't need to provide a file. Use the file format defined in the configuration. Follow the existing settings for the separator character, escape character, and other formatting rules. You can't change these settings when running the import. Check the File Format Information section to understand the format before running the import.
3	Optional: If you have the `sys.sync-bypass-threshold` right, you can bypass the import thresholds.
4	Optional: Click Simulate to preview the import without applying changes.
5	Click Run to start the import. If you want to schedule the import for later, click Plan (top right) and choose a date and time. The system will run the import once at the scheduled time. It won’t repeat automatically.

Visualize an Import Task - Public Import

Only users with the appropriate rights can access export files generated by public (scheduled) imports.

	Steps	Preview
1	To view the details of a public import: Go to the Public Imports section. Find the task you want to review. Click Details to view task information and access export files (if you have the required permissions).

Steps

Preview

To view the details of a public import:

Go to the Public Imports section.
Find the task you want to review.
Click Details to view task information and access export files (if you have the required permissions).

Access Control

You can grant execution rights on a per-task basis. For example, if Import Tasks A and B exist, you can allow a user to execute Task A while restricting access to Task B.

You can also assign administrative rights to let a user execute any Import Task.

The system checks permissions every time a user tries to run an import task.

These permissions follow standard access control schemes:

Permissions

Permissions	Scope	Description
`ImportTaskAccess:private`	TASK (per Synchronization Task Definition)	This permission is based on the `SynchronizationTaskDefinition` resource. This means that a User can have this permission to access task executions for some `SynchronizationTaskDefinition`s only, but not others. With this permission, a user can run, view, and cancel private task executions they launched (whether immediate or scheduled).
`Sync:run Sync:monitor`	GLOBAL	Execution of any `SynchronizationTaskDefinition`. Retrieval of any `SynchronizationTaskDefinition` execution report. These are global permissions. They apply to all import tasks, not just specific ones.

Permissions

Scope

Description

ImportTaskAccess:private

TASK

(per Synchronization Task Definition)

This permission is based on the SynchronizationTaskDefinition resource. This means that a User can have this permission to access task executions for some SynchronizationTaskDefinitions only, but not others.

With this permission, a user can run, view, and cancel private task executions they launched (whether immediate or scheduled).

Sync:run Sync:monitor

GLOBAL

Execution of any SynchronizationTaskDefinition.

Retrieval of any SynchronizationTaskDefinition execution report.

These are global permissions. They apply to all import tasks, not just specific ones.

You can grant users the following rights, each controlling a specific level of access:

Rights

Right	Permissions	Security Scope	Description
sys.sync-import-task-private	`ImportTaskAccess:private`	TASK (per Synchronization Task Definition)	Give capacity to run a Synchronization Task Definition and create a new private Task execution.
sys.sync-bypass-threshold		GLOBAL	Allows to do not enforce thresholds
sys.sync-progress		GLOBAL	Allows to display progress bar
sys.sync-task-run	`Sync:run`	GLOBAL	Run all Synchronization Task Definitions.
sys.sync-task-admin	`Sync:run` `Sync:monitor`	GLOBAL	Run or retrieve all executions of Synchronization Task Definitions.

The private task execution permission that allows end users to retrieve an import report is specific to the Import feature (see next section).
This type of private permission doesn’t exist for regular Synchronization Tasks.

Configuration

Don’t confuse the execution of an Import Task with its configuration.

Administrators configure the tasks, and the end-users execute the tasks if they have the appropriate execution rights on the Import Task. In case of a scheduled import, the system executes the task.

An Import Task is configured through a classic Synchronization Task. When you execute an Import Task, the system actually runs a Synchronization Task behind the scenes. The "import" aspect comes from the ability to upload a CSV file on-the-fly when the user launches or schedules (once) the Synchronization Task.

The core object remains the Synchronization Task, which must reference the following components to work properly:

Application. This is the main configuration entry point. It defines the mapping between IDM service attributes and the remote system's attributes.
- which itself references a connector, the component enabling Synchronization Task to communicate with the remote application
Post Synchronization Task (optional). Defines what happens after the sync, such as post-processing or reporting actions.

Example

Synchronization Task Configuration Example

XML

<dmn:SynchronizationTaskDefinition id="importTask-partner" xmlns:dmn="http://www.memority.com/domino_sync/1_0">
      <name>Import Partner</name>
<description></description>
  <active>true</active>
  <allowCyclicInputData>false</allowCyclicInputData>
  <allowDuplicateInputData>false</allowDuplicateInputData>
  <applicationId>import-partner-application</applicationId>
  <bypassIdmDeduplication>false</bypassIdmDeduplication>
  <bypassIdmReferenceIntegrity>false</bypassIdmReferenceIntegrity>
  <bypassIdmUnicity>false</bypassIdmUnicity>
  <bypassIdmValidation>false</bypassIdmValidation>
  <objectClassId>__ACCOUNT__</objectClassId>
  <recoverable>false</recoverable>
  <shadowKind>ACCOUNT</shadowKind>
  <simulate>false</simulate>
  <skipCorrelationOnImport>false</skipCorrelationOnImport>
  <sortInputData>false</sortInputData>
  <taskType>IMPORT</taskType>
</dmn:SynchronizationTaskDefinition>

REST API

The Memority Synchronization service exposes the following Import REST API:

Full import REST API

Resource	Method	Body	Permission	Description	Response
All APIs under `{tenant}/api/sync/~run-sync-task` relate to instances management and require a `Sync:run` permission
`{tenant}/api/sync/{task-definition-id}/instances/~content`	POST multipart	JSON + FILE	Sync:run	Launch an import task by providing the identifier of an `SyncTaskDefinition` task configuration (path variable), an `SyncTaskRunOptions` object (JSON payload with key options) and a data file to import (FILE payload with key file) The execution of this method is asynchronous, it immediately returns to the caller an UUID, the task instance id, that can then be used in the API to get an execution report.	Success HTTP code 200. Body: a task instance id as a simple string (not JSON): an UUID identifier enabling to monitor the task progress, or cancel the running task (see below). Errors HTTP code 404 if the task configuration id is not found.
`{tenant}/api/sync/conf/sync-task-definitions/{task-definition-id}/~data-format`	GET	N/A	Sync:run	Return information about the expected CSV file format.	Success HTTP code 200. Body: Information about the expected CSV file format: header names, mandatory fields, fields format.
All APIs under `{tenant}/api/sync/~private/~run-sync-task` relate to private instances management and require an `ImportTaskAccess:private:?` permission
`{tenant}/api/sync/~private/{task-definition-id}/instances/~content`	POST	JSON + FILE	`ImportTaskAccess:private:{task-definition-id}`	Launch an import task by providing the identifier of an `SyncTaskDefinition` task configuration (path variable), an `SyncTaskRunOptions` object (JSON payload with key options) and a data file to import (FILE payload with key file) The execution of this method is asynchronous, it immediately returns to the caller an UUID, the task instance id, that can then be used in the API methods below.	Success HTTP code 200. Body: a task instance id as a simple string (not JSON): an UUID identifier enabling to monitor the task progress, or cancel the running task (see below). Errors HTTP code 404 if the task configuration id is not found.
`{tenant}/api/sync/~private/{task-definition-id}/instances/{task-instance-id}/~cancel`	POST	N/A	`ImportTaskAccess:private:{task-definition-id}`	Cancel a running task.	Success HTTP code 200 Body: empty. Errors HTTP code 404 if the task instance id is not found. HTTP code 409 ("conflict") if the task cannot be canceled because it is in a terminal state.
`{tenant}/api/sync/~private/{task-definition-id}/instances/{task-instance-id}`	GET	N/A	`ImportTaskAccess:private:{task-definition-id}`	Get the execution report of a task.	Success HTTP code 200 Body: JSON representation of the task execution report. Errors HTTP code 404 if the task instance id is not found.
`{tenant}/api/sync/~private/instances/`	GET	N/A	`ImportTaskAccess:private:{task-definition-id}`	Get all the visible execution reports. Reports can be filtered by passing the following optional HTTP parameters in the GET request: `filter` : string, part of the name (case insensitive) of a Task Definition whose reports are retrieved `kind` : choice among IDENTITY, ORGANIZATION OR RESOURCE, reports for this kind of objects are retrieved `expandErrorDetails` : boolean, if true all errors encountered during the import task execution are returned, not only a subset of the 20 first errors
`{tenant}/api/sync/conf/sync-task-definitions/~private/{task-definition-id}/~data-format`	GET	N/A	`ImportTaskAccess:private:{task-definition-id}`	Return information about the expected CSV file format.	Success HTTP code 200. Body: Information about the expected CSV file format: header names, mandatory fields, fields format.

Options can be provided in the JSON body when launching an import task, they are listed below:

Property Name	Type	Mandatory	Description	Values (default value in bold)
executionPlan	`Complex`	NO	Only set when the Task execution is deferred to a later date. This is the CRON expression defining when the task will be executed (only once). It can be executed only once, thus providing a recurrent CRON expression is forbidden.	-
simulate	`boolean`	NO	Whether the task execution is simulated or not.	false, true

Here is an example of a multi-part curl command used to launch an Import Task with a CSV file and the "simulate" option set to "true":

Launch an import task

curl -i -X POST \
-H "Content-Type: multipart/form-data" \
-u login:password \
-F "options={\"simulate\":\"true\"};type=application/json" \
-F "file=@mydata.csv" \
http://localhost:8150/memority/api/sync/~run-sync-task/myimportTask/~content

Here is an example of JSON returned by ~data-format:

Data format

{
  "metadata" : {
    "type" : "CSV",
    "encoding" : "utf-8",
    "uniqueAttributeId" : "uid",
    "quote" : "\"",
    "escape" : "\\",
    "commentMarker" : "#",
    "fieldDelimiter" : ",",
    "recordSeparator" : "\r\n"
  },
  "attributes" : [ {
    "attribute" : "age",
    "mandatory" : false,
    "multiValued" : false
  }, {
    "attribute" : "email",
    "mandatory" : false,
    "multiValued" : false
  }, {
    "attribute" : "login",
    "mandatory" : true,
    "multiValued" : false
  }, {
    "attribute" : "nom",
    "mandatory" : true,
    "multiValued" : false
  }, {
    "attribute" : "prenom",
    "mandatory" : true,
    "multiValued" : false
  }, {
    "attribute" : "societe",
    "mandatory" : true,
    "multiValued" : false
  }, {
    "attribute" : "titre",
    "mandatory" : true,
    "multiValued" : false
  }, {
    "attribute" : "uid",
    "mandatory" : true,
    "multiValued" : false
  } ]
}