Peregrine HPC Energy Research

Complete power and performance metric data for jobs run on the Peregrine system in April, 2015. Format: The data is available as a compressed JSON stream — one line per JSON object. Each object is a hash containing three elements, each of which is itself a hash: “power” “ganglia” “job” The “job” object contains metadata for the job including these fields: id unique identifier for this job user_name user running the job — anonymized group_name group running the job — anonymized account allocation the job is charged to — anonymized queue the queue the job is submitted to: https://hpc.nrel.gov/users/systems/peregrine/running-jobs/new-queue-structure submit_time start_time end_time exit_code exit code from Moab nodelist colon separated list of node hostnames used for the job nodes_used processors_used wallclock_used torque_wallclock_used torque_exit_code exit code according to torque cpu_used mem_used vmem_used wallclock_req feature_req nodes_req processors_req feature requests interactive whether or not the job was run interactively node_hours app_name application category app_name_predicted whether or not application category was predicted using a Naïve Bayes classifier. The "power" and "ganglia" objects contain time-series data from iLo http://www8.hp.com/us/en/products/servers/ilo/ and NWPerf https://github.com/EMSL-MSC/NWPerf/. The format of the timeseries data is a list of hashes containing, timestamp (epoch time in microseconds), metric name, host and value, e.g.,: { "timestamp": 1461697084441, "host": "n1668", "value": 228, "metric": "power" } Anonymized fields have been SHA1 hashed.

Complete power and performance metric data from 10,000 jobs randomly selected from two years of the Peregrine system’s operation (all jobs prior to October, 2015). Format: The data is available as a compressed JSON stream — one line per JSON object. Each object is a hash containing three elements, each of which is itself a hash: “power” “ganglia” “job” The “job” object contains metadata for the job including these fields: id unique identifier for this job user_name user running the job — anonymized group_name group running the job — anonymized account allocation the job is charged to — anonymized queue the queue the job is submitted to: https://hpc.nrel.gov/users/systems/peregrine/running-jobs/new-queue-structure submit_time start_time end_time exit_code exit code from Moab nodelist colon separated list of node hostnames used for the job nodes_used processors_used wallclock_used torque_wallclock_used torque_exit_code exit code according to torque cpu_used mem_used vmem_used wallclock_req feature_req nodes_req processors_req feature requests interactive whether or not the job was run interactively node_hours app_name application category app_name_predicted whether or not application category was predicted using a Naïve Bayes classifier The "power" and "ganglia" objects contain time-series data from iLo http://www8.hp.com/us/en/products/servers/ilo/ and NWPerf https://github.com/EMSL-MSC/NWPerf/. The format of the timeseries data is a list of hashes containing, timestamp (epoch time in microseconds), metric name, host and value, e.g.,: { "timestamp": 1461697084441, "host": "n1668", "value": 228, "metric": "power" } Anonymized fields have been SHA1 hashed.