Sicurezza per i motori remoti con DataStage Anywhere

Panoramica

Quando si esegue un flusso DataStage, il flusso viene compilato come codice Orchestrate Shell (OSH). Al runtime, il codice OSH viene convertito in dati JSON e viene inviato al motore remoto in cui si verifica l'esecuzione del lavoro. Il motore remoto si trova all'interno del VPC, quindi le pipeline di dati possono essere eseguite dietro il firewall sicuro, senza esporre alcun dato al piano di controllo. Il cliente mantiene la piena riservatezza dei dati in conformità con le normative vigenti in materia di sovranità. Quando un progetto DataStage è legato all'esecuzione di un motore remoto, non è possibile tornare a Cloud Pak for Data as a Service, poiché ciò potrebbe introdurre vulnerabilità di sicurezza.

Comunicazione

DataStage utilizza le chiamate API per comunicare tra il piano di controllo e il piano dati. La comunicazione va sempre dal motore remoto al piano di controllo, comprese le chiamate API e il protocollo di comunicazione di baseTLS1.2). Ogni istanza del motore remoto ha una propria chiave di crittografia (algoritmo di crittografia AES 256) che viene generata automaticamente per le connessioni e le credenziali dei dati.

I registri vengono trasmessi a IBM Cloud per tracciare l'utilizzo degli entitlement (VPC), ma sono disponibili anche dove si trova il motore remoto.

Esempi di comunicazione del piano di controllo e motorizzazione remota

Il motore remoto inserisce il tipo di lavoro Compilare:

[
                    {
                        "work_type": "compile",
                        "metadata": {
                            "status_id": "ds.runtime:flow.compile.status.details:a8fe3d27-ddea-4b10-8c04-c687a5eeb3b4:e2ed740f-51e6-41b6-80d1-80838085d3b5",
                            "project_id": "a8fe3d27-ddea-4b10-8c04-c687a5eeb3b4",
                            "flow_id": "e2ed740f-51e6-41b6-80d1-80838085d3b5"
                        },
                        "work": {
                            "px_executables": "UEsDBBQACAgIAJWTi1gAAAAAAAAAAAAAAAAoAAAAZTJlZDc0MGYtNTFlNi00MWI2LTgwZDEtODA4MzgwODVkM2I1Lm9zaO1Ya1PbOBf+7l/hXd4Zt7NxHOce43iHQtnSUqAE2AtlPLKtBBVHdmWZyzL973skx8TGcSjT3Z2deUsI6PLonKNzk3Q2NjY21N0wulFZdDPDVPejxPWj+M41B6Zpds2OOeiM+g9D7UF/1BqYI7Ot+tE8JiEOVMRVc2i1h1ZnqJqmuhUztd1qd5WNb/2RFNTJydYvry31OLpxf8EUM8Qj5powpx7GWU+Z5ePFUTWKOYloouiJf4nnSGXYj1igvFBUdftw//T9gWtaCWeEzs7NVuticznerhnv1Ix3a8Z7NeP9mvFBzfiwZnxUM262yhMvFT3bfKKaLUURWspUGT4o6ZwEmPIX2iM1ay831U+RN4+oWw9IOJrhfP7oFhAPAO3lhbRJyuOUJ0rLUc8v1Mc0rH1Cr1yzea0pm3+z15wwRJNpxOb4sc/wfGa1z0xDNFNZShWdojlWtRIhTVHWa7GMruiwMl3S4HZheiImFkrco5kO7Sd0WNV3iV+GbAukWTvd+SeMsffqvbsdRmngHnqfsM/dCYguNl62THzrMRLM8ErDKPohI6AmJLqZ3hT9t/f7RywCKCc4UTX759t5qF5jlgBm/FEzm62PmoqpHwUQEzBwerKrm30Y+9mxCwuLK8yPmmNvR3OwmvhPOb7lKr+LMcwSymG2bRuLccc+Q4wg+gDIgg8wwNk2FpOOvYMTn5EYtnOWcVqNr8BASAQSih2fwIKyGLppG6Vpxz7ePipjgCiMOULgbEOnCejNsb3Uv8JVqe0fzrd3tk62zn1hLD2SxtKTzFjihIA2ogFigZ5+Hl1cAOGMkmO/RdfIsd9gFE/In48kfSDb7vXlohwGzWzdDg7JnHAcTO7AwreO/TmNOHZpCp5J/CQn50VRWKRnSmplrGNPCUu4GxKK3UuMAszqlrfk8grcsXOPqVdQyqf6UC7PsY6w36NdTOGYdGUaqSUUII6afnK9EGWBd+w4tyycs2vFLwAd+1d0jd+AhUIp0FYcYxqc7u2sp/AAg3aZwA2D3bjzKKgzaEZgCXNsCn+FLbyl2leuKeAc22cYAYWyV64WtgRdsPOjlPIabuaSm4QtjCKSHlq/poATcYPZdoiS5GCtNeFq1CQefEnS9P2mDKMsihZB1JRpMMuCiyQomZXoy76MU8gzFJAyFRA0d1MW1jO/5DxOLMMAYJPjhGfspTwglyEPGcLvDB5dYSq5Lkg6NoqJe4Xvctoxg4jywZGrTO6DhELmA6//snv0ip68G8S7x9OYXE3OfjJeH37Y5x+OD/p/DJPfY3Y0m7waXg0OeTv43dya97wPuP/u3dbN2cElP6YnUoQFZ8dmOIlS5sNhSEWSEY1gjaIZta7FLQ7NYMaKUy8kvrUqbVmzMPJQaCHD88xRe4h6nSDArWHfH/RxHwWoNRj0MBpNkdXp9vxBr9/S+50+1qHj6cOuh/QpGnb6QzTCI69nWVLsVdI69lrzJJ1mmug3D4Z5JGdmMhTHQTRHhGYYyUtaCBzkeMFzb8GyPq7N3KeqS+RZsHQqY3kKOpoCKV5OwbGr3SvX2fGVn6MNJSQeQ+xu/OMnSNqW9CkPviQxfN+o+rpR9fX9jELjfy92Jm8O379+aTSbxs4ETqcYEhjlieERCtQSH/yi+QmxzaeQK0Ls6WX8krBAJM67MhB+tyavDiBZGNgPSZxgIw5T8DDYYTGyQ3EREUNwpiZ6GM2EE+pms13l/TySbyeHB923QAk0Dp+ndyIV9S0cRdLImnq32fv2HUh6EYPwaYrPOmrgTobwpAdFkkhvP7HmGWaBYxn7QpUrKFb0+HZvshP58uTD7Ksc72sdVOxQfw64DNz8saHk11NwikYelSFJRGSemZswalnfQ/J7SH4PyX8rJMVhuAjKRRwuTkhxdx9X/V/5osH7Fe7UV+UHK0dsBhfe1jNerv/uw/UYi13ANSFv/OdesvlNuXCRUcVNZqnae2VnMvmwL+iO7/N63ths5CWzZbO9bHaXzd6y2V82B8vmcNkcFei2xuaXRsb7iGGfSEMVBGi1CiIsO+1ip1vs9IqdfrEzKHaGxc6oxKcleplQsjIK2q3WRutqnc+tmf7na6MfNRGXa4p59WWramVvPbZU5qvmh6PfVlb66qp3f3d5boI/p+JtiEJXPnrLVTl8G0eMf02BXVXvp4QClUAUQsaYBg1VFmfGQQQPNNxQswmtoTVUmoaCHQ6DsXZwur+vffk/LdALlauaeP11RCEIclcEaV3WVET1XrqJCg8zTmiKnyg/VyxZddSVkEeF/CVmFyBf65uL0vFfUEsHCPBDvuS5BgAAWhoAAFBLAwQUAAgICACVk4tYAAAAAAAAAAAAAAAAFQAAAFRyYW5zZm9ybWVyXzEudHJ4Lm9zaOWT30vDMBDH3/NX3NOqjGirc/5iT6uCMO3YJgpjlJBeu2CTlEsK+/OtbDr2EMVH8eXIfXL3TfIlxzwJ40pLGrgyTeudXKMWQCgtFeyIAYyzyfPjU57cOE/KVMskjle3e34W4OcBPgjwiwAfBvhlgF8F+HWAJ/HhxjFjjNvWf+dF6G2/9egPePGTFf/nW3DcNITOKWtKVSMse+l8+jq3LUlMFa1OF5+jhJQnJ542jJe1qEBa3XQNjO8WBFHV70dfuW08RFwCz4CX04cx8BdjeYHdaVJ4LIDr4aALvjU4qtAgqa5YS20LrEdOi7ruxGpl3vbSH9lW160F7TQ6boRGOLgo44Ui6M0W+X02Se9m71BLBwjc1e+EDgEAABIEAABQSwMEFAAICAgAlZOLWAAAAAAAAAAAAAAAABEAAABUcmFuc2Zvcm1lcl8xLnRyeJ2Qz07DMAzG75X6Dj7CqS27ICreAO0wJK4otJ5mSOPJcTYJxLvjpCvaJnHhkj9f7N/3xU0DI24pIHASoLBP2nBS28BT+IDgJox1VR7yGVp4Mv216+tqrjtX7y7VblZXpmYEKTlPnwhfdQXNhfHAQYU9HJyQe/PZ0sLoPWz4uMF3HBTHtl/EdfL+GfXFSVvQ33mZHAXPvM/0mb84RgTdIciJA8LHXyOrPLeAR+j6pR8PzienaOl8mkKEm8A5aVQxL423sGUpY3o4fd/6jkKKggPLCCXxFb5o/4GvruDdH/AyDBssxV0ehV1/AFBLBwg/DVWC6gAAAOQBAABQSwMEFAAICAgAlZOLWAAAAAAAAAAAAAAAABMAAABvc2hfZGVwZW5kZW5jeS5qc29uq1bKyymOz00siAfROfnJiTmpSlbVSpm5BakVBfHJGYlFxaklSlZKoSFuuhZKOkrFJUWZeelYJDLzCkpLsIjnl5Zgk6jVUcpISiwGWhYdWwsAUEsHCFZRERFRAAAAhgAAAFBLAQIUABQACAgIAJWTi1jwQ77kuQYAAFoaAAAoAAAAAAAAAAAAAAAAAAAAAABlMmVkNzQwZi01MWU2LTQxYjYtODBkMS04MDgzODA4NWQzYjUub3NoUEsBAhQAFAAICAgAlZOLWNzV74QOAQAAEgQAABUAAAAAAAAAAAAAAAAADwcAAFRyYW5zZm9ybWVyXzEudHJ4Lm9zaFBLAQIUABQACAgIAJWTi1g/DVWC6gAAAOQBAAARAAAAAAAAAAAAAAAAAGAIAABUcmFuc2Zvcm1lcl8xLnRyeFBLAQIUABQACAgIAJWTi1hWURERUQAAAIYAAAATAAAAAAAAAAAAAAAAAIkJAABvc2hfZGVwZW5kZW5jeS5qc29uUEsFBgAAAAAEAAQAGQEAABsKAAAAAA==",
                            "ds_flow_environment": {
                                "FlowEnvironment": {},
                                "FlowParameters": {}
                            }
                        }
                    }
                ]

Il motore a distanza tira nel tipo di lavoro Esegui, esempio 1:

{
    "work_type": "job_run",
    "metadata": {
        "project_id": "a8fe3d27-ddea-4b10-8c04-c687a5eeb3b4",
        "job_run_id": "69f6a270-6912-4321-a913-29277628eed1",
        "job_id": "aee11bf2-5be7-4abd-8930-1b0bbba4c6dd",
        "accept_language": "en-US,en;q=0.5"
    },
    "work": {
        "FlowEnvironment": {
            "S3_BUCKET": "testauth-donotdelete-pr-rs1hqmcbhwuzud",
            "AWS_ID": "4102d83e9d19412ca4ffa6a585f7b80b",
            "S3_REGION": "us-south",
            "AWS_KEY": "9aede861b3cb994bfdc605af10d58c2cf87ccb8e8c8d005c",
            "NUM_PARTITIONS": "2",
            "$REMOTE_ENGINE": "2220b98e-4f80-49d8-827c-371168a0d920",
            "S3_ENDPOINT": "https://s3.us-west.cloud-object-storage.test.appdomain.cloud",
            "HARDWARE_SPECIFICATION": "{'datastage':{'compute':{'mem':{'size':'16Gi'},'cpu':{'model':'','units':'4'}},'num_conductors':1,'num_computes':0,'conductor':{'mem':{'size':'4Gi'},'cpu':{'model':'','units':'1'}}}}"
        },
        "FlowParameters": {
            "DSJobRunName": "",
            "job_name": "rowgen-cos.DataStage job",
            "flow_name": "rowgen-cos",
            "project_name": "Test-Auth",
            "DSMetricsRepoEnabled": "false"
        },
        "FlowMessageHandler": {},
        "FlowOsh": "UEsDBBQACAgIAJykiFgAAAAAAAAAAAAAAAAoAAAAYjc0OTE1OGItMjI3OS00Nzg4LTgyY2MtNjE2MDdmYTk2OWYwLm9zaO1YW1PbOBR+96/wlp1JmcV2nBvBGHcCKS0UCBBg2wXGI9tKEDiWKylclul/3yPZJgm5UKY73X0oISDJn75zjs5FspaWlpb07Zje6Yze9XFihJTrIR2kJMaRjoReKTvVhlMt6+Wm3koZ9Cs1belHfxSD3j1pfXjv6Mf0zv+AE8yQoMy34ZneSbOe1i/Gx0d1mgpCE64ZPLzCA6QzHFIWaW81Xd/q7J3uH/i2wwUjSf/cLpcv10fjlTnj1TnjtTnj9cnxZc3IdOC6XdY0qWxmUfyk6zmJcCLelp5ZW1pe169pMKCJPx/ABerj4vnhPSCeAKXlS7U0Q5EOBdfKnn5+qT/ncPZIcuPb5m1JW/+Xnbezue9vxXQY+Z3gGofC74JEqe2kH9P7gJGoj2e6UTM6jIBtSHYzYzXj8/7eIaMAFQRzveS+ux/E+i1mHDAbFyXbLF+UdJyENAI3wMDpybZhN2DsneeOTRyfYV+UPHeLDmCx5f9E4Huhi4cUw1OSCHhaca183HPPECMoeQJk/gYMSHat/KHntjEPGUnBnLNM0mz8FAyURKChtPgEJkyqYdiuNfHYc4+3DicxQApjnlQ4M+iUw7p5bjAMb/C01u5v51vt1knrPJTOMqhylsEzZ8mshzZKIsQiY/h17fISiDMmz91Ft8hzP2KUdsnfzzR9oq3UG2pSAYNmNq+NYzIgAkfdB/Dwved+HVKB/WQ4wIyEvKALKI3H+WzFNon13B5hXPgxSbB/hVGE2bzpZTV9Cu65RcTMX6Ch6BlNNb3AetJ/z6zoQYH0EzTA84kiJJAZ8ttclRzvuWnhWaiwC9UfA3run+gWfwQPxUqhVpriJDrdaS9meIJBe5LgjoE1/oBG8xyaEYxgnpvAX+mLYLTsM+eM4Tw3ZBgBw2RUzlZ2ApqLC+kwEXOk2SNpCpY7pUfZAC2eM4aTeYPZVow4P1joTdgUTRLAl3AzDE2VRlkW5UlkqjKYVcG8CCphE/yqr/IU6kwCSFUKCBr4QxbPF34lRModywKgKTAXmXilD+hlqZ2BiAdL0BucKKk5peeilPg3+KHgThlkVAiBPC3kMeIJVD6I+m/bh5vJyafVdPu4l5Kb7tkf1vvO0Z44Oj5o/NXkX1J22O9uNm9WO6ISfbFbg3pwhBufPrXuzg6uxHFyolTIJXsuw5wOWQg7WCKLjGxECxaaJc6t3L9RH5446TCISejMKltOP6YBih1kBYG9VmmiejWKcLnZCFcbuIEiVF5drWO01kNOtVYPV+uNstGoNrABncBo1gJk9FCz2miiNbwW1B1HqT1LW89d6B5eNYfcuHtyzDM9M5ehNI3oAJEkwyhZykMQIMe5zJ1c5Py8touYmp6i9oJRUFmjXdAraVDi1SPYdkuP2m22fRX76IoWk4Ah9rDx5hqKtqNiKoAv4VYYWtOxbk3H+l7GsPL723b3Y2f//bJlmla7C7tTCgUsEdwKSAJsPIS4MK8RW38JOSPFXp4mrgiLZOF8mATCb6u7eQDFwsJhTFKOrTQeQoSBheOZHcuDiByCPZUbMe3LIDRsszIt+3WUu93OQW0XmGDF4fOyJWqhfkSiLBpZ06iZ9R+3QPFRBuljys8iNggnS0bS00ISalRemPMKt8C2jEO5lDMYp9Zxd6fbpqHa+TD7rsD73gCVFhqvAU8C19+saMXxFIJipcjKmHCZmWf2Oow6zq+U/JWSv1LyZ6Wk3AzzpMzzMN8h5dl9Yzr+tW8leH+FM/XN5AurQKwPB97yK95cf+6L6zGWVsAxoWj8795ki5Py2EFGlyeZ0dI+au1u92hP8m48Fjc/G/ZKcdkzalZHzdqoWd+wv61kHIcMh0Qt+BhRuTxGNepUxzu18U5ddjJKdTMFNv73d1MXJRmkC66k5t/hTN9OLcZOXFRNJ8vh5/y6aifJbqvcF2+r/gFQSwcIHQDnraoFAAChFAAAUEsDBBQACAgIAJykiFgAAAAAAAAAAAAAAAATAAAAb3NoX2RlcGVuZGVuY3kuanNvbqtWysspjs9NLIgH0Tn5yYk5qUpW1UqZuQWpFQXxyRmJRcWpJUpWSqEhbroWSjpKxSVFmXnpWCQy8wpKS7CI55eWYJOo1VHKSEosBloWHVsLAFBLBwhWURERUQAAAIYAAABQSwECFAAUAAgICACcpIhYHQDnraoFAAChFAAAKAAAAAAAAAAAAAAAAAAAAAAAYjc0OTE1OGItMjI3OS00Nzg4LTgyY2MtNjE2MDdmYTk2OWYwLm9zaFBLAQIUABQACAgIAJykiFhWURERUQAAAIYAAAATAAAAAAAAAAAAAAAAAAAGAABvc2hfZGVwZW5kZW5jeS5qc29uUEsFBgAAAAACAAIAlwAAAJIGAAAAAA==",
        "FlowResourceRequirements": {
            "Core": 0,
            "Storage": 0,
            "Memory": 0,
            "Partitions": 0,
            "Compute": 0
        },
        "FlowLimits": {
            "WarnLimit": 100,
            "RowLimit": 0
        },
        "FlowId": "b749158b-2279-4788-82cc-61607fa969f0",
        "Secrets": [],
        "FlowProjectSettings": {
            "runMessageHandlerId": "",
            "runPriorityQueue": "Medium",
            "formatDateString": "",
            "formatTimestampString": "",
            "formatTimeString": "",
            "formatDecimalSeparator": "",
            "nlsMap": "",
            "collationSequence": "OFF"
        }
    }

Il motore remoto tira nel tipo di lavoro Esegui, esempio 2:

{
    "work_type": "job_run",
    "metadata": {
        "project_id": "a8fe3d27-ddea-4b10-8c04-c687a5eeb3b4",
        "job_run_id": "bfdea00d-6ede-45b6-8cf7-540f17e520ee",
        "job_id": "eb9de08a-37ef-4c47-9591-abe4c4a57d1b",
        "accept_language": "en-US,en;q=0.5"
    },
    "work": {
        "FlowEnvironment": {
            "S3_BUCKET": "testauth-donotdelete-pr-rs1hqmcbhwuzud",
            "AWS_ID": "4102d83e9d19412ca4ffa6a585f7b80b",
            "S3_REGION": "us-south",
            "AWS_KEY": "9aede861b3cb994bfdc605af10d58c2cf87ccb8e8c8d005c",
            "NUM_PARTITIONS": "2",
            "$REMOTE_ENGINE": "2220b98e-4f80-49d8-827c-371168a0d920",
            "S3_ENDPOINT": "https://s3.us-west.cloud-object-storage.test.appdomain.cloud",
            "HARDWARE_SPECIFICATION": "{'datastage':{'compute':{'mem':{'size':'16Gi'},'cpu':{'model':'','units':'4'}},'num_conductors':1,'num_computes':0,'conductor':{'mem':{'size':'4Gi'},'cpu':{'model':'','units':'1'}}}}"
        },
        "FlowParameters": {
            "DSJobRunName": "",
            "job_name": "rowgen-cos_copy_1711141317396_copy_1712769071912.DataStage job",
            "flow_name": "rowgen-cos_copy_1711141317396_copy_1712769071912",
            "project_name": "Test-Auth",
            "DSMetricsRepoEnabled": "false"
        },
        "FlowMessageHandler": {},
        "FlowOsh": "UEsDBBQACAgIAKyKilgAAAAAAAAAAAAAAAAoAAAAZTJlZDc0MGYtNTFlNi00MWI2LTgwZDEtODA4MzgwODVkM2I1Lm9zaO1YW1PbOBR+96/wlp1JmcVxnHuMcQdIaaFAgADbLjAe2VaCwLFdSeGyTP/7Hsl2EuMklOluZx9KCMjSp+8cnZtkraysrKg7QXSv0uh+iEPNi5jjRfGjY7QMw6gbNaNV6zQnXdVWs1NpGR2jqnrRKCYB9lXEVaNlVg3TaKpGRd2MqVqtVOvKyo/+SAa1f7r54b2pnkT3zgccYop4RB0DxtRenDwpw6x/tleNYk6ikCka867xCKkUexH1lbeKqm739s8ODh3DZJyScHhhVCpX69P+6oL+2oL++oL+Rr5/VdESHRjYSVGEssmKgomuF8THIX9berba0uq6ehO5oyh0FgMYR0OcjR89AGICKK1eSdOMeTzmTKnY6sWV+pzD3CfhrWOU70rK+r/svFOKQjaI6Ag/dx3PRua7bhCgoUrHoaKFaITVUo6opCjLrZhHF2xYGM5ZcHtmuC8GUiPuhokNrRdsWLR3Tl6CrP4X1t7dOnC2g2jsOz33Bnvc6YNuYmV508cPLiX+EM+1vKL1KAE7IPGYGEbRPh/sH9EIoJxgppasdw+jQL3DlAFm47JklCuXJRWHXuRD0EPH2emOZjSh751tzUycnWFclmxrOxqBW8T/kOMHrvLHGMMoCTmMVi097betc0QJCieAJLsAA5ItPR20rS5mHiUxLOc8kTQfX4CBkgg0FCs+hQl5NTTD0nPDtnWyfZTHACn02ULhZEFnDOxmW+7Yu8VFra3fLra7m6ebF55wlhZJZ2kscZaoxNBGoY+or42/dq6ugDhhsq09dIds6yNGcZ/8/UzTCW210ZSTMhg0k3ldHJAR4djvP4KHH2zr6zji2AnHEJnEYxmdG0XBLJ8h2fJY2xoQyrgTkBA71xj5mC6aXpHTC3DbyiJmsYHGfKC15fQMawv/PVvFALYjR9aJhUQ+4qjssbtUlRRvW3HmWdjPlqo/A7StP9Ed/ggeCqRCm3GMQ/9st7ucYQKDdp7gnsJqnFHkL3JoQjCF2VYIf4Uv3KnZ586ZwdmWRzEChnxUzlc2B03FedE45AukGVNpEpY6RRQ9tHzODE7kDabbAWLscKk34QhSJi58CSt7XlmmUZJFaRKVZRlMqmBaBKWwHL98lnkKdSYEpCwFBI2cMQ0WC7/mPGamrgOwzDHjiXipD+ily12E8EedR7c4lFJTSttCMXFu8WPGHVPIKA8CuSjkyWchVD6I+m87R1vh6adWvHMyiMlt//wP/X3veJ8fnxw2/2qzLzE9Gva32retHq/6X4zNUcM9xs1Pnzbvzw+v+Ul4KlVIJdsWxSwaUw92u1AUGdHwlxiahuadOC2hIYyY8dgNiGfOK1vmMIhcFJhId12jU22jRs33caXd9FpN3EQ+qrRaDYw6A2TW6g2v1WhWtGatiTV4cLV23UXaALVrzTbq4I7bME2p9jxtbWupe1itPGba/cQxz/RMXIbi2I9GiIQJRsqSHoIAOUll7qYiF+e1kcVUcYrcC6ZBpU93QbukQImXQ7Dtlp6Uu2T7yvbRNSUgLkX0cePNDRRtU8aUC1/CdM/Ti7GuF2N9P2FY+/1tt/+xd/B+VS+X9W4fdqcYCljIme6SENiYB3FRvkF0/SXknBR7eRq/JtQXhfMxD4Tfzf7WIRQLHXsBiRnW42AMEQYrnM3sQBxERBfsqUwLoqEIQs0oV4uyX0e51+8d1veACSwOn5dXIg31IxJF0UiaWr3c+PEVSL6IQvqUxWcZG4STLiJpYkgSadUX5rzCLbAtY0+Ycg5jwY57u/1u5MmdD9PvCrzvDVCxQu014Dxw/c2akh1PISjWsqwMCBOZeW6sQ69p/krJXyn5KyV/VkqKzTBNyjQP0x1SnN03ivGvfCvB+yucqW/zL6wc0SEceCuveHP9uS+uJ1isAo4JWeN/9yabnZRnDjKqOMlMTfukdPv9433Bu/GU3ZttGGvZldu0WZ0269NmY8P4tpZwHFHsEWnwGaJKZYZq+lCdfajPPjTEQ0Ip7wFhjcWbwEU3e6+9Ifzem8DLkgjSJVdXi+9wivdYy7G5S61ishx9nnuvtfiu6h9QSwcIxiK7RwoGAAAzFgAAUEsDBBQACAgIAKyKilgAAAAAAAAAAAAAAAAVAAAAVHJhbnNmb3JtZXJfMS50cngub3NolZBPS8NAEMXv+ynm1ChltNHqQempURCqKW1FoZSwbCbp4v4Jsxvoxzdibekhopdh5sfMe48RkaULlWcLqF3TxqC2ZCUwKc+lOBMA03z2+vxSpHchsnb1Oh2NNvdHftXDr3v4uIffnPJzIQT6Nv4Wqc/iv1H/HAlp1zCFoL2rtCFYD7Ll/H3pW1aUad5crn6+SVykF5F3Aisja1DeNt2BwH3DkNTDYXKYfRMhQQWYA1bzpyngm/NYUuemZKQS0N6OuxJbR5OaHLHulq2yviQzCVYa04kZ7T6O0l/Tt27YSt5rdNxJS3ASVGCpGQaLVfGYz7KHxSdQSwcIKooocfUAAAAVAgAAUEsDBBQACAgIAKyKilgAAAAAAAAAAAAAAAARAAAAVHJhbnNmb3JtZXJfMS50cnhdkM9OwzAMxu+V+g4+wqkdJ0TFGyAOm8QVhdbTDG48Oc4qgXh3nGxFg0v+fLG/3+d0HUy4p4ggWYHiMVsn2XwDpvgBMcyY2qY+lDP08OT662Zom3PdtXrnaikmo8D0ifDVNtD9QYwSTYXhFJTCGxdzx9o9bGXZ4juOhlM/rOJzZt6hvQTtq/V3WeZAkUWOxf3svxITgh0Q9OIDKssvyCuvEfAIm2Htx1PgHAw9Hec5JriJUpImU2dZuoW9aP2Qh8ug3rcoGSqOohPUxP/si1bz+uyUDiWtX38AUEsHCAOe9m/ZAAAAcQEAAFBLAwQUAAgICACsiopYAAAAAAAAAAAAAAAAEwAAAG9zaF9kZXBlbmRlbmN5Lmpzb26rVsrLKY7PTSyIB9E5+cmJOalKVtVKmbkFqRUF8ckZiUXFqSVKVkqhIW66Fko6SsUlRZl56VgkMvMKSkuwiOeXlmCTqNVRykhKLAZaFh1bCwBQSwcIVlEREVEAAACGAAAAUEsBAhQAFAAICAgArIqKWMYiu0cKBgAAMxYAACgAAAAAAAAAAAAAAAAAAAAAAGUyZWQ3NDBmLTUxZTYtNDFiNi04MGQxLTgwODM4MDg1ZDNiNS5vc2hQSwECFAAUAAgICACsiopYKooocfUAAAAVAgAAFQAAAAAAAAAAAAAAAABgBgAAVHJhbnNmb3JtZXJfMS50cngub3NoUEsBAhQAFAAICAgArIqKWAOe9m/ZAAAAcQEAABEAAAAAAAAAAAAAAAAAmAcAAFRyYW5zZm9ybWVyXzEudHJ4UEsBAhQAFAAICAgArIqKWFZRERFRAAAAhgAAABMAAAAAAAAAAAAAAAAAsAgAAG9zaF9kZXBlbmRlbmN5Lmpzb25QSwUGAAAAAAQABAAZAQAAQgkAAAAA",
        "FlowResourceRequirements": {
            "Core": 0,
            "Storage": 0,
            "Memory": 0,
            "Partitions": 0,
            "Compute": 0
        },
        "FlowLimits": {
            "WarnLimit": 100,
            "RowLimit": 0
        },
        "FlowId": "e2ed740f-51e6-41b6-80d1-80838085d3b5",
        "Secrets": [],
        "FlowProjectSettings": {
            "runMessageHandlerId": "",
            "runPriorityQueue": "Medium",
            "formatDateString": "",
            "formatTimestampString": "",
            "formatTimeString": "",
            "formatDecimalSeparator": "",
            "nlsMap": "",
            "collationSequence": "OFF"
        }
    }
}

Il motore remoto invia lo stato del lavoro, esempio 1:

{
    "Job Status": "RUN",
    "HB_INSTANCE_HOST_URI": "",
    "Project Id": "a8fe3d27-ddea-4b10-8c04-c687a5eeb3b4",
    "Queue End": 1712608618451,
    "px-instance-id": null,
    "send-running-count": 1,
    "Job Startup Time": "0",
    "Flow Run Id": "69f6a270-6912-4321-a913-29277628eed1",
    "message-type": "JobStatus",
    "Queue Start": 1712608618449,
    "msg_count": 0,
    "Flow Id": "aee11bf2-5be7-4abd-8930-1b0bbba4c6dd",
    "Job Production Run Time": "0"
}

Il motore remoto invia lo stato del lavoro, esempio 2:

  {
    "Job Status": "FOK",
    "HB_INSTANCE_HOST_URI": "",
    "job_metrics": {
        "check_end": 1.712608634152849E9,
        "start_end": 1.712608618652393E9,
        "finished": 1.712608650705897E9,
        "total_stages": 2,
        "link_metrics": [
            {
                "start_time": 1.712608634E9,
                "rows_read": 10,
                "rows_written": 10,
                "stop_time": 1.712608634E9,
                "source": "Row_Generator_1",
                "state": "finished",
                "dest": "IBM_Cloud_Object_Storage_1",
                "link_name": "Link_1"
            }
        ],
        "total_seconds_cpu": 21.15,
        "run_end": 1.712608650705897E9,
        "duration": 32.11314010620117,
        "total_rows_read": 10,
        "check_begin": 1.712608618652393E9,
        "run_begin": 1.712608634152849E9,
        "stage_metrics": [
            {
                "stage_seconds_cpu": 21.15,
                "start_time": 1.712608648E9,
                "rows_read": 10,
                "rows_written": 0,
                "stop_time": 1.71260865E9,
                "stage_type": "cloudobjectstoragePX",
                "stage_name": "IBM_Cloud_Object_Storage_1",
                "state": "finished"
            },
            {
                "stage_seconds_cpu": 0.0,
                "start_time": 1.712608634E9,
                "rows_read": 0,
                "rows_written": 10,
                "stop_time": 1.712608634E9,
                "stage_type": "PxRowGenerator",
                "stage_name": "Row_Generator_1",
                "state": "finished"
            }
        ],
        "job_id": "360",
        "start_begin": 1.712608618592757E9,
        "state": "finished",
        "total_rows_written": 10
    },
    "Project Id": "a8fe3d27-ddea-4b10-8c04-c687a5eeb3b4",
    "Queue End": 1712608618451,
    "px-instance-id": null,
    "send-running-count": 7,
    "S3_LOG_LOCATION": "DataStage\/69f6a270-6912-4321-a913-29277628eed1.job.log",
    "Job Startup Time": "0",
    "Flow Run Id": "69f6a270-6912-4321-a913-29277628eed1",
    "message-type": "JobStatus",
    "Queue Start": 1712608618449,
    "msg_count": 7,
    "Flow Id": "aee11bf2-5be7-4abd-8930-1b0bbba4c6dd",
    "Job Production Run Time": "32"
}

Il motore remoto invia le metriche di esecuzione del lavoro:

{
        totalCpu: 4000
        usedCpu: 0.0
        totalMemory: 2014576640
        usedMemory: 115087176
        totalDisk: 250059685888
        usedDisk: 160418521088
        engineInitialized: 1712608520755
    }
    computes: null
}

Dati sensibili

Per evitare di scrivere dati nei registri, evitare di utilizzare le funzioni Peek Stage o Asset Browser. Invece, utilizzate i file sequenziali per analizzare i dati e i tipi di dati effettivi utilizzati nella progettazione del lavoro e per parametrizzare tutti i nomi dei file e le connessioni.