Conversation
Adding fio-specific pmlogger.cfg and openmetrics config files
Switch the results files (interim and final) from colon to comma separated values to support OpenSearch integration
|
Please check the format of the csv file, appears the header for the csv is out of place. It is appearing in the meta data, not with the block of data. |
dvalinrh
left a comment
There was a problem hiding this comment.
The pcp header is
o.w.iterat o.w.runnin o.w.numthr o.w.runtim o.w.throug o.w.latenc o.w.ndisks o.w.njobs o.w.iodept o.w.operat o.w.blksiz o.w.bw o.w.iops o.w.clat o.w.slat o.w.lat
This will cause the backend tool to have knowledge of what the values of operation etc is. Better solution is to have that identifying information in pcp metric, similar to uperf
o.w.iteration o.w.running o.w.numthreads o.w.runtime o.w.throughput o.w.latency o.w.streams_test o.w.rr_test o.w.biderec_test o.w.maerts o.w.numb_networks o.w.tcp_packet o.w.udp_packet o.w.packet_size o.w.number_procs o.w.GB_Sec o.w.trans_sec o.w.lat_usec
it will make the file a bit larger, but it places all the relevant test information in the log, and will make it easier to take this data and convert into the dbase (they donot have worry about what the value in the field means, just if it is 0 it is not used, if > 0 it has meaning.
Change from "1-10" logging of operation type to a 0/1 metric for each potential type. Supposed to be better for downstream database and AI work
|
Adding updated pcp-openmetrics output post-code revew |
Description
This PR adds logging of fio "settings knobs," run results, and some disk usage metrics to the run's PCP archive
This PR updates the results.csv format from colon to comma separated value and adds timestamps to support opensearch integration
Before/After Comparison
Before:
No logging of results and disk metrics with PCP
Sample from results.csv: op:
blocksize_KiB:njobs:ndisks:iodepth:bw_KiB_s:iops:clat_us:lat_us:slat_us
read:1024:1:1:1:2058648.875000:1996:126277:456:500:43
After:
Results are logged with PCP
op,blocksize_KiB,njobs,ndisks,iodepth,bw_KiB_s,iops,clat_us,lat_us,slat_us,Start_Date,End_Date
read,1024,1,1,1,2058648.875000,1996.126277,456,500,43,2026-01-28T13:41:27Z,2026-01-28T13:43:33Z
Clerical Stuff
This closes #56
This closes #57
Relates to JIRA: RPOPC-807
Relates to JIRA: RPOPC-808
Supporting docs:
results_fio.csv
pcp_openmetrics.txt
pcp_disk.txt
fio.out.txt