Data mapping for Service Graph Connector for AWS

Data from the AWS data sources is mapped and transformed into the ServiceNow CMDB Configuration Item (CI) class definitions using the Robust Transform Engine (RTE). Data is inserted into the ServiceNow® CMDB using the Identification and Reconciliation Engine (IRE).

Data mapping for AWS

Data from the AWS data sources is mapped and transformed into the ServiceNow CMDB Configuration Item (CI) class definitions using the Robust Transform Engine (RTE). Data is inserted into the ServiceNow CMDB using the Identification and Reconciliation Engine (IRE).

Note: If the Use last run datetime field for an AWS data source in the Data Source [sys_data_source] table is empty, the connector imports all available initial data. If the Use last run datetime field includes a date stamp, the connector imports incremental data that has been newly added since the previous run.

The following table lists the import schedule order, the data sources and import schedules of the same name, the staging tables, the target tables as CMDB CI classes, the import schedule requirement type, and the import schedule dependencies for AWS.

For more information on where data is saved when pulling data from AWS, see CMDB classes targeted in Service Graph Connector for AWS and Supported AWS resource types.

You can use the IntegrationHub ETL app to view the data maps. See IntegrationHub ETL for more information.

For more information about how CI information is pulled from AWS, see the Service Graph Connector for AWS - Functional Spec and CI article on the ServiceNow Community site.

Additional information about AWS data sources:

• The SG-AWS-Generic-Resources data source imports data for generic resources that aren't tracked by other data sources. The connector uses the Service Graph Resource Inclusion Whitelist [sn_cmdb_int_util_service_graph_resource_inclusion_whitelist] table to differentiate between generic and other supported resource types.

  The connector first populates all supported resources in the Service Graph Resource Inclusion Whitelist [sn_cmdb_int_util_service_graph_resource_inclusion_whitelist] table. These resources, categorized under their respective supported resource types, have specific data sources designated for ingestion within the connector. When the SG-AWS-Generic-Resources data source is executed and retrieves unsupported resource types, they are added to the Service Graph Resource Inclusion Whitelist [sn_cmdb_int_util_service_graph_resource_inclusion_whitelist] table and categorized as generic.

• In Service Graph Connector for AWS version 2.13.0 and later, parallel loading is enabled for SG-AWS-Generic-Resources data source if the AWS Config aggregator is used for discovery. Partitions are created for the SG-AWS-Generic-Resources data source based on the Resource Types in the Service Graph Resource Inclusion Whitelist [sn_cmdb_int_util_service_graph_resource_inclusion_whitelist] table. Additionally, delete jobs aren't created for a full data load.

  If the AWS Config aggregator isn't used for discovery, API calls are made to multiple accounts and the Partition info field is populated with Account and Region details.

• To import global generic resources such as IAM user and IAM group, specify a standard AWS region that has Config enabled with includeGlobalResourceTypes set to true by updating the value of the sn_aws_integ.global_generic_resource_region system property for the SG-AWS-Generic-Resources data source.

  If an aggregator is configured, and the sn_aws_integ.global_generic_resource_region value is not specified, the aggregator region is assigned as the value of this system property for importing global generic resources.

• If the AWS Systems Manager (SSM) service isn't enabled, the connector populates the server records in the Server [cmdb_ci_server] class. If the AWS SSM service is enabled, then based on the platform type obtained through the SSM service, the server records are populated in either the Linux Server [cmdb_ci_linux_server] class or the Windows Server [cmdb_ci_win_server] class. The Server [cmdb_ci_server] class is the parent class of the Linux Server [cmdb_ci_linux_server] and the Windows Server [cmdb_ci_win_server] classes.

• All labels associated with an AWS resource are added to the Key Value [cmdb_key_value] table.

  Note: You can use the CMDB Data Manager to delete tag data from retired CIs in the Key Value [cmdb_key_value] table based on conditions like retention time and discovery source. A scheduled job runs the policy, which can be configured to execute during off-peak hours.

• The basic information about an AWS resource is stored in the SG-AWS Extension Attributes [sn_aws_extension_attributes] table.

• In Service Graph Connector for AWS version 2.10.0 and later, the SG-AWS-Get-Inventory data source runs before the SG-AWS-EC2 data source and creates a Server [cmdb_ci_server] CI with the host name mapped to the Name attribute, instead of being mapped to the VM name.

• The SG-AWS-GenericTags data source imports tag data only for generic resources that have an ARN key. You can use the SG-AWS Extension Attributes [sn_aws_extension_attributes] table to verify which generic resources have an ARN key.

  Note: The SG-AWS-GenericTags data source doesn't import tag data for Amazon CloudFront resources.

• The AWS configuration data for each connection is stored in the SG AWS Application Properties [sn_aws_integ_sg_aws_application_properties] table.

• When you run the diagnostic test, the data is loaded in the following tables:
  • SG AWS Diagnostic Details [sn_aws_integ_sg_aws_diagnostic_details]
  • SG-AWS Diagnostic Summary [sn_aws_integ_sg_aws_diagnostic_summary]
  • SG AWS Diagnostic Summary Notes [sn_aws_integ_sg_aws_diagnostic_summary_notes]
• In Service Graph Connector for AWS version 2.13.0 and later, the SG-AWS-Image-Id data source doesn't access cross-account records during lookup.

Related content

CMDB classes targeted in Service Graph Connector for AWS

Service Graph Connector for AWS properties