5.7.2. TaskStatistics¶

Description¶

Statistical data of a task and thumbnail download links for the specified period of time. A similar method is used for data plotting and thumbnails timeline visualization the Boro interface (the task page).

For positioning in the timeline data, time from a task start is used. In other words, at the very start of a task the time value is 0, and then it increases as the analysis is being performed. This approach facilitates unification of Live and Offline (VoD, file) tasks statistics, when the analysis is faster or slower than this in real time. Actual time is the point in time at which the latest statistics are collected.

Scheme of data request for different positioning commands position_type:

Data per one second¶

The probe collects statistical data on a 1-second interval and then transmits it to the server. Last 300 seconds of the received data are considered to be hot and stored in the cache. It helps significantly speed up the loading of initializing data. Old data (data over 300 seconds) is stored as single database records, each record comprises data for 5 minutes. Access to such data can be much slower than to the hot data. Due to the mentioned storing feature, data statistics may be returned for a larger period than it was requested.

Data scaling¶

Under development: the probe also collects data for the following intervals: 10, 100 and 1000 seconds.

Request¶

Important

To get some metrics, it is necessary to enable certain monitoring options and properly configure the thresholds profile.

Note

Data for no more than 3000 seconds can be returned in reply to one request.

{
  "user_id":(number),
  "methods":[
    {
      "method":"TaskStatistics",
      "params":{
        "project_id":(number),
        "task_id":(number),
        "position_type": (string),
        "time": (number),
        "duration": (number),
        "metrics": [(string)]
      }
    }
  ]
}

user_id - an integer value, a user identifier;
project_id - an integer value, a user project identifier;
task_id - an integer value, a task identifier;
position_type - a string, a command for positioning in the timeline data:

init - to get initializing data for the last 300 seconds (unless another period for the duration field is specified). It may be used for quick visualization of the current state of parameters after page refreshing;

updated_since - to get all the data following after the specified time value until the actual time (without considering duration, but no more than 3000 s). It may be used for periodic updating of graphs which are plotted on the basis of data returned by the init command;

around - to get the data preceded by and following after the specified time value (300 s are in total, unless another period for the duration field is specified). It may be used when the required event is considered to be in the center of a graph. With the help of this method, positioning the required event in the graph center in the Boro interface is implemented. It is performed by clicking the event in the journal on the task page;

after - to get the data following after the specified time value (300 s, unless another period for the duration field is specified). It may be used for timeline data history visualization;

before - to get the data preceded by the specified time value (300 s, unless another period for the duration field is specified). It may be used for timeline data history visualization.
time (optional field) - an integer value, time in seconds from a task start. It is used in several positioning types position_type for specifying the starting point. In the reply, the max time which is present in the returned statistics for the requested interval. The init command allows to get the actual task time.
duration (optional field) - an integer value, a requested data interval in seconds. The limit of duration is 3000 s. It is used for specifying a requested data interval in relation to a starting point for all positioning types position_type except updated_since.
metrics (optional field) - an array of strings, selection of required metrics. If no metrics are specified in the request, all metrics data will be returned in the reply. The detailed metrics description is in the documentation of Elecard Boro Service:

EPSNR - Estimated Peak Signal to Noise Ratio, expressed in dB for each video stream;

Bitrate - bitrate of each PID (bps) detected in a stream;

DownloadRate - Download Rate or Multicast Rate depending on the type of the analysed source. It is measured in bps;

IAT_MDI - combines three metrics related to IPTV network streaming: Maximum Inter-packet Arrival Time (maxIAT) (ms); Media Lose Rate (MLR) - number of lost transport packets; Delay factor (DF) - time value indicating how many milliseconds’ worth of data the buffers must be able to contain in order to eliminate time distortions;

AudioLoudness - combines two metrics related to audio loudness measurements in accordance with the EBU R 128-2011 standard: Momentary Loudness and Short-Term Loudness registered in LUFS;

VideoDecodability - estimation of video decodability based on ratio of the number of correctly decoded frames to declared framerate;

Thumb - thumbnails download links for each video stream;

CC - number of Continuity Counter errors in a transport stream per one second (according to ETSI TR 101 290 p1.4);

ClockDiscontinuity - PTS/DTS Discontinuity, number of errors per second.

Reply¶

Note

Since the database stores statistical data as 5-minute records, in the reply statistics may be returned for a larger period than it was requested. You will get as many 5-minutes records from the DB as it is required to cover the requested interval. If necessary, additional filtering of statistical data may be performed on the client side.

Features and exceptions¶

The method returns data for every metrics as a table wrapped in JSON. The data structure of EPSNR and Bitrate metrics implies that values for some PIDs cannot be skipped; the server uses the null stuffing value if the probe skips data for a certain PID for some reasons.

If for the requested interval there is no metrics data (the monitoring option is disabled), or no errors (e.g. CC errors) are registered, the reply will not include the corresponding fields. The DownloadRate metric is always present in the reply, even if the task is in the BadSource (No Signal) state.

When the task enters the BadSource (No Signal) state:

For the Bitrate metric, the value null is sent once for all PIDs and further, while the error status is active, data is absent. When the probe receives the first correct input data, statistics will appear in the metrics;
For the IAT_MDI metric for maxIAT, MLR and DF, the value null is constantly present, while the state is active;
For the DownloadRate metric, zero speed is returned.

{
  "reply":[
    {
      "method":"TaskStatistics",
      "result":{
        "head":{
          "time":(number),
          "data_version":2
        },
        "EPSNR":{
          "cols":["time","PID1","PID2", ... , "PIDn"],
          "data":[
            [1021523.8250531789,50.890653318618305,50.69966307076358, ..., 49.81497413611472],
              ...
          ]
        },
        "Bitrate":{
          "cols":["time","PID1","PID2", ... , "PIDn","tei"],
          "data":[
            [1021523.8249982789,15280,12224,... ,25977,565651],
              ...
          ]
        },
        "DownloadRate":{
          "cols":["time","rate, bps"],
          "data":[
            [1021524.6812821408,10779843],
              ...
          ]
        },
        "IAT_MDI":{
          "cols":["time","maxIAT","MLR","DF"],
          "data":[
            [1021524.4743567407,71.849,0,72.03139948869963],
              ...
          ]
        },
        "AudioLoudness":{
          "cols":["time","PID","momLoudness","stLoudness"],
          "data":[
            [1021523.825054879,1022,-24.895469665527344,-22.834897994995117],
            [1021523.825054879,1072,-25.58980941772461,-23.23800277709961],
              ...
          ]
        },
        "VideoDecodability":{
          "cols":["time","PID","realFPS","statFPS","declaredFPS"],
          "data":[
            [1021523.825054879,1021,25,25,25],
            [1021523.825054879,1071,24.0416667461395264,24.041667103767395,25],
              ...
          ]
        },
        "Thumb":{
          "cols":["time","url","id","pid"],
          "data":[
            [1021524.1780905407,"241555/thumbs/112163793?ut=1589269172",112163793,1021],
            [1021525.6247920685,"241555/thumbs/112163796?ut=1589269173",112163796,1071],
              ...
          ]
        },
        "CC":{
          "cols":["time","CC","[PID, count]"],
          "data":[
            [1021534.0012250001054852,1,[[1071,1]]],
            [1021574.6247920685,12,[[1022,4],[1072,3],[1021,3],[1071,2]]],
              ...
          ]
        },
        "ClockDiscontinuity":{
          "cols":["time","ClockDiscontinuity","[PID, count]"],
          "data":[
            [1021574.6247920685,2,[[1021,1],[1071,1]]],
              ...
          ]
        }
      }
    }
  ]
}

time - a real number, the max time which is present in the returned statistics for the requested interval. Executing the init command allows to get the actual time of the task;
EPSNR - an object, Estimated Peak Signal to Noise Ratio expressed in dB for each video stream:
- time - a real number, a data timestamp;
- PIDn - a real number, the EPSNR value in decibels (dB). The field name (column) is the PID number in decimal format.
Bitrate - an object, the bitrate of each PID (bps) detected in a stream:
- time - a real number, a data timestamp;
- PIDn - an integer, the bitrate value in bps. The field name (column) is the PID number in decimal format;
- tei - an integer, the bitrate (bps) of packets with the Transport error indicator (TEI) set.
DownloadRate - Download Rate or Multicast Rate depending on the type of the analysed source:
- time - a real number, a data timestamp;
- rate, bps - an integer, the bitrate value in bps.
IAT_MDI - an object, three metrics related to IPTV network streaming:
- time - a real number, a data timestamp;
- maxIAT - a real number, Maximum Inter-packet Arrival Time (maxIAT) in milliseconds;
- MLR - an integer, Media Lose Rate (MLR) - number of lost transport packets;
- DF - a real number, Delay factor (DF) - time value indicating how many milliseconds’ worth of data the buffers must be able to contain in order to eliminate time distortions.
AudioLoudness - an object, two metrics related to audio loudness measurements in accordance with the EBU R 128-2011 standard:
- time - a real number, a data timestamp;
- PID - an integer, an audio stream PID in decimal format. If a stream contains several audio streams, data arrays will be interleaved;
- momLoudness - a real number, Momentary Loudness in LUFS;
- stLoudness - a real number, Short-Term Loudness in LUFS.
VideoDecodability - an object, three metrics, estimation of video decodability based on ratio of the number of correctly decoded frames to declared framerate:
- time - a real number, a data timestamp;
- PID - an integer, a video stream PID in decimal format. If a stream contains several video streams, data arrays will be interleaved;
- realFPS - a real number, the number of correctly decoded frames per second;
- statFPS - a real number, average framerate for the last 30 seconds;
- declaredFPS - a real number, declared framerate.
Thumb - an object, download links for thumbnails for each video stream:
- time - a real number, a data timestamp;
- url - a string, a relative URL for thumbnails downloading. An absolute URL example:
  
  http://172.16.11.111/en/projects/23/apps/702/tasks/241555/thumbs/112163793?ut=1589269172
- id (not used) - an integer, a thumbnail identifier;
- PID - an integer, a video stream PID in decimal format. If a stream contains several video streams, data arrays will be interleaved.
CC - an object, Continuity Counter errors:
- time - a real number, a data timestamp;
- CC - an integer, the total number of Continuity Counter errors in a transport stream per one second (according to ETSI TR 101 290 p1.4);
- [PID, count] - an array of arrays, information on errors for each PID:
  
  PID - an integer, a stream PID in decimal format;
  
  count - an integer, a number of errors;
ClockDiscontinuity - an object, PTS/DTS errors:
- time - a real number, a data timestamp;
- ClockDiscontinuity - an integer, the total number of PTS/DTS errors in a transport stream per one second;
- [PID, count] - an array of arrays, information on errors for each PID:
  
  PID - an integer, a stream PID in decimal format;
  
  count - an integer, a number of errors.

Example¶

Request by the cURL utility¶

#Initializing data request for all metrics
  curl http://172.16.11.111/ctrl_api/v1/json \
   -H "Content-Type: application/json" \
   --data '{"user_id":4,"methods":[{"method":"TaskStatistics", "params":{"project_id":23,"task_id":241555, "position_type":"init"}}]}'

Visual representation of data returned in the reply in the Elecard Boro interface (the task page):

Reply to the request¶

 {
   "reply":[
     {
       "method":"TaskStatistics",
       "result":{
         "head":{
           "time":1021823.7122620306,
           "data_version":2
         },
         "EPSNR":{
           "cols":["time","1021","1071","1091"],
           "data":[
             [1021523.8250531789,50.890653318618305,50.69966307076358,49.81497413611472],
             [1021524.8253254407,50.89207579602636,50.69161823423854,50.05298566057196],
               ...
             [1021821.9385350284,46.640793752131,47.68373994934159,47.479309657171044],
             [1021822.9390992803,44.76247796973997,48.69961482382366,48.231185218245415]
           ]
         },
         "Bitrate":{
           "cols":["time","0","16","17","18","1020","1021","1022","1024","1070","1071","1072","1090","1091","1092","1130","1132","7000","7010","8191","20"],
           "data":[
             [1021523.8249982789,15280,12224,25977,87100,15280,2716921,198650,38201,15280,2715393,198650,15280,2716921,197121,15280,200178,1528,13752,1002418],
             [1021524.8252731407,14923,8953,23876,85061,14923,2717499,196985,37307,14923,2717499,198477,14923,2717499,198477,14923,201462,0,16415,1005818,1492],
               ...
             [1021821.9384111284,14925,8955,25373,86566,14925,2716403,198506,37313,14925,2717896,198506,14925,2716403,198506,14925,199998,0,16417,1004472,1492],
             [1021822.9390142803,14868,11894,25276,87725,14868,2718005,199241,38658,14868,2716518,197754,14868,2718005,197754,14868,200727,1486,14868,999179]
           ]
         },
         "DownloadRate":{
           "cols":["time","rate, bps"],
           "data":[
             [1021524.6812821408,10779843],
             [1021525.6813746685,10009549],
               ...
             [1021822.7123416803,10009588],
             [1021823.7122620306,10009753]
           ]
         },
         "IAT_MDI":{
           "cols":["time","maxIAT","MLR","DF"],
           "data":[
             [1021524.4743567407,71.849,0,72.03139948869963],
             [1021525.4748181686,71.926,0,72.21568719710159],
               ...
             [1021822.1680089284,71.896,0,71.9357831331915],
             [1021823.1688340803,72.461,0,72.52942528068608]
           ]
         },
         "AudioLoudness":{
           "cols":["time","PID","momLoudness","stLoudness"],
           "data":[
             [1021523.825054879,1022,-24.895469665527344,-22.834897994995117],
             [1021523.825054879,1072,-25.58980941772461,-23.23800277709961],
             [1021523.825054879,1092,-41.76763153076172,-38.99705505371094],
             [1021523.825054879,1132,-23.177711486816406,-23.156808853149414]
               ...
           ]
         },
         "Thumb":{
           "cols":["time","url","id","pid"],
           "data":[
             [1021524.1780905407,"241555/thumbs/112163793?ut=1589269172",112163793,1021],
             [1021525.6247920685,"241555/thumbs/112163796?ut=1589269173",112163796,1071],
             [1021527.3863328784,"241555/thumbs/112163797?ut=1589269175",112163797,1091]
               ...
           ]
         }
       }
     }
   ]
 }