Real-time data displays

We seem to have gone backward this year in the provision of reliable and timely match data. In previous seasons we have had the TCS website which although not perfect was pretty good for following matches. Additionally many league websites had a summary of latest scores albeit somewhat delayed depending on whether the information was passed back to league via text or email.

My league has abandoned their own website and has moved everything over to Play-Cricket where some of the latest scores are displayed but since quite of few of the teams don't use PCS there are omissions.

For those scores that are supplied by PCS there are a number of Play-Cricket widgets and displays but none of them really work correctly.

The problems I've noticed this year are:

The live score widget can be set to display matches for a number of days, in most cases clubs only want to display the current match so set the duration to be days=0. This is fine for one day matches but a 3 day match will not display in the widget on day 2 or 3.

None of the widgets are 'real time'. My background is in really real-time data where microseconds count but for the sake of this discussion we can call real-time data something that arrives on the display at most 1 second or so after the event happens - runs scored, wicket taken etc.
The Play-Cricket widgets operate on a polling basis - that is they retrieve the data every 3 minutes which isn't great if you are trying to follow an exciting finish.

You can follow a match by looking at the 'results' section of Play-Cricket if the scorer is using PCS and acting as a live scorer. However, once again this is polled rather than event driven so you will find yourself constantly refreshing the page to get the latest score. If you are viewing the display on a mobile device it becomes incredibly frustrating in the second innings because the default page load condition is to hide the second innings which requires you to scroll down and tap to expand the panel to see the scores. In a 3 day scrolling to the bottom to see the last innings is even more frustrating.

 

 

Does this drive anyone else up the wall? An innings is in progress and every batsmen who has not batted is marked as 'did not bat'. This is a consequence of somebody not understating how the Play-Cricket API works. Oh, and why can't we have a total at the bottom of the scorecard for an innings?

So what can we do about this?

Firstly we need to change the underlying mechanism used to display the data. Page refreshing went out of date in about the year 2000 there are so many better ways of doing things now - primarily delivering data to a browser as soon as the data changes.

The following videos are probably best viewed full screen. They are on YouTube so whatever video comes next after viewing mine - that is nothing to do with me!


What does this look like?

Here we have a web page which updates almost immediately after the score has been updated in PCS and we only update the data that has changed. It is easy to switch between innings.

All the displays you will see in these demonstrations can be embedded within your own club website or used as standalone web pages.

Responsiveness is a buzzword that you will hear often - basically it means that a web page will scale to the size of the screen of the device you are using. Normally this means that the layout will change or the font size will be reduced. What we can also do is use responsiveness to change the data being displayed - in this case on larger screens we can display the full names of the players - this is fully automatic.

One of the irritations of Play-Cricket scorecards is that you never know if the display has stopped updating because something is broken or whether play has stopped for some reason.

Our real time display will tell you immediately.

We can also utilise the event driven delivery of data to put some real-time charts and gauges into webpages. This is a bit of fun, the meter displays the current run rate, if the pointer is in the green area then it is above the required run rate, yellow - a bit iffy and red seriously behind the required rate.

We do miss the good old worms and run-rates charts from TCS - so now we can add them to our websites. Of course all these charts update in real-time.

 

 

 

 

We can also get very creative in making charts that can be used for team and player analysis and although these charts update in real-time their real value is for understanding how a match went and how players performed.



Here we display the duration of a batsman's innings, and the percentage of the team's runs.

Dot ball analysis in limited over matches is a bit of black magic pseudo science but we can add to it with another chart. Here we display the overs bowled, the percentage of dot balls and in the last column the percentage of wickets taken.

We can also add charts to analyse how a batsman has performed against the bowlers and how a bowler performs against the batsmen.
Firstly the bowler v the batsmen.

And the batsman v the bowlers

Case study - Using the CRI file

One of the joys of attending a cricket match is when there is a scorecard available.
Mobile phones are all well and good but there is nothing like a printed copy to read and analyse and if the mood takes you, actually fill in the scores.

You can generate a bit of income for your club by selling them or by printing adverts on the back of the card.

We can use the PCS export (cri file) to generate the scorecard for a match.
A working utility is available at http://pcsutil.thehomeofcricket.com

Unfortunately the cri export doesn't include the full names of the players (until they have batted or bowled), only the scoreboard name, so make sure you scoreboard names are correct in the PCS database.

Click on Scorecard and then upload your CRI exported file and click on Process File.

If a team logo already exists in the database it will be used, if not you may add a logo.

 

 

 

You do not have to have a logo it simply makes the scorecard more attractive.

Click Generate Scorecard and you will be presented with a printable version like this.

Technical Details

This section may be of interest to programmers. It demonstrates how you can load the CRI file into an XDocument and then extract the data required to build the card.

The only potential gotcha is that the 'Competition' may not exist, if this is the case we have to get the information from the MatchType stanza.

 
           

 

Yacadu Updates

I've updated Yacadu to include the match officials and the 'highlights' in the upload.

Currently the import doesn't load the match officials into TCS - I'm working on that but if you select the officials whilst using TCS they will be uploaded to Play-Cricket.

The comments text box on the upload form is sent to P-C as 'Highlights'.
I have no idea if there is any limit to the number of characters that P-C will accept but I assume it is at least a couple of sentences.

Remember that if you want to use Play-Cricket 'live scores' you can upload your match with the result type set to "Match In Progress"

Yet another half finished project

Some years ago I had the idea of being able to reproduce a 'Hawkeye' system on a single domestic camcorder. I believed that it would work but wasn't absolutely sure until I produced a prototype.

To be able to do this requires some fairly complex 3D mathematical transforms and it is an awfully long time since I used anything like that level of maths so it was a struggle. Added to which the video processing and encoding technology was way behind what we have today but more or less it works.

This is one the analyses.

 

 

All we need to know here is the height of the stumps (pretty much standard) and the distance between the stumps - 22 yards for adults. We can then create a transformation matrix to convert our model of what the ball is doing to the real world view from the camera - in effect we are determining the viewing point height and distance, zoom and hither and yon planes in 3D from a single 2D camera.

Once we know this matrix we can transform any 2D point - such as the bowler releasing the ball into our 3D matrix, thus by selecting a couple of points on the video, typically release point and pitch point we know exactly what the ball is doing. We have to of course add in some basic projectile theory and equations, an estimate of the coefficient of restitution of the mat (how much the ball will bounce), and some assumptions about air resistance.

From this we can produce the Hawkeye predicted path and the actual pitch map.

One day I will restart this project because I think it has huge potential as a coaching aid.

Live Video

I have been playing around with producing live video multiplexed with the scoreboard output of PCS Pro.

To do this properly you really need professional grade cameras but these cost tens of thousands of pounds so I've been rummaging through my old kit boxes to see what works.

I started with a pair of webcams, one with a mobile phone telephoto lens attached similar to this:

https://www.amazon.co.uk/Universal-Telescope-Telephoto-Camera-Samsung/dp/B01J3Q6JQA/ref=sr_1_15

The result is surprisingly effective for an outlay of £20 or so, but not really good enough.
Here is the video.

https://youtu.be/Q398F5Plhd8?t=1h33m54s


Last weekend I resurrected and old Sony Handycam (the one with tapes) with a proper Carl Zeiss lens. We are now into the realm of optical rather than digital zoom and the results are a significant improvement.
I didn't quite get the aspect ratio and canvas size correct which accounts for the rather tall players.

https://youtu.be/bvZeTb9fYcI?t=5h55m30s

 

So how do we put all this together?

There are a number of sources involved
1 The scoreboard output from PCS Pro
2 The video stream
3 The audio stream. I never have this switched on, the last thing I want to be broadcasting is the conversations on the balcony!

We also need a bit of software and service.

YouTube account.
Copy of Open Broadcaster Software - this is free and open source.
A piece of software to turn the PCS scoreboard output into a bitmap.


We also require piece of hardware to turn the analogue AV signal from the camera into digital, these are commonly known as USB video grabbers and cost about £10.

In OBS we add the video stream, the bitmap source, our YouTube account details, click the Start Streaming button and we are live broadcasting so all your members can view the match wherever they are.

My old Sony also has a IEEE 1394 (FireWire) interface which in theory should provide much higher quality video since the encoding is done in hardware on the camera rather than in software on the laptop. Unfortunately my only laptop which has FireWire is from the Windows XP generation and isn't man enough for the job of broadcasting. I will be hunting around for an alternative over the coming season.

Next weekend I hope to get the aspect ratio correct and I will update with the results.

The High Wycombe live feed can be found at

https://www.youtube.com/channel/UCD5m18VOaY4Kywgk8wHP_zg

What is a CRI file?

There appears to be some confusion about the PCS import/export file format.

When you export a match a sub-directory is created in the target directory containing a file with the name formatted as home v away + date.CRI

Many people have asked if they can use this file in the same way as the TCS export for putting data on their own websites. The answer is yes, but it requires a little more work.

The first thing to understand is that the format of a CRI file is XML.
There have been suggestions that one day there will be a CRI to XML translator - this of course is daft, it is not required, at least in the basic sense.

Anyone who has tried to open one will find that it is unreadable via a text editor or browser. This is because the file is compressed using a mechanism called gzip, short for GNU zip - an open source technology.

There are many utilities available to decompress gzipped files, for example,  WinZip, 8 Zip and a command line version from gzip.org.

Having decompressed the file and added an XML file extension you will be able to open it in a text editor eg notepad or a browser. Note that each XML element doesn't have a nice line feed or indentation so you may want to use an XML editor. The little utility which comes with Windows 10 called Code Writer is good for this.

What you will see is something like this:

All data entities in PCS Pro have an identifier known as a GUID ( globally unique identifier) which is simply a stonkingly big number. It uses 128 bits of storage and is almost guaranteed to be globally unique if not universally unique.

The use of GUIDs as identifiers of database records is considered to be very bad practice. For each record you are carrying an overhead of 128 bits rather than  the more usual 16 or 32 bits, however it is unlikely that your PCS database is going to get very large even over several years of scoring and the GUID does have the advantage of being portable - you can import a match into an existing database without, for example, any reasonable fear of there being any duplicate players.

In a subsequent post I will explain what you can do with the XML data.

A better live score widget

This was deleted from the PCS google group, perhaps someone took exception to me describing the Play-Cricket live score widget as clunky and a stealer of real-estate on the screen.

Let me explain -
In the default configuration the 'widget' gets all the matches for a club for the day on which you are viewing it.
Typically this will be a Saturday and if you have 5 teams playing then it will display 5 latest score panels.

From the ECB website here is an example-

You can configure the widget to show only one team or a league but this has to be hard coded, it is far better to ask for all the matches and display only those that have data.

Now, my club fields 5 teams only two of which have scorers and at times not even both of them will be 'live scoring'. This means that there are up to 4 panels taking up space on the screen which will never update. That is disregarding any colts matches which may also be taking place and will certainly not have live scores. It is possible that both the 1s and 2s will be live scoring so I want a generic widget to only display those who have live scores and not lose valuable space to dormant matches.


Update:
The is some horrid logic in how the Play-Cricket widget works. There is a parameter to the URI
&days=n

Unfortunately you have to use this is you want a 3 day match displayed  - if days=1 then on the second day the match will be displayed but not on the third day. If days=0 (the default) then the match will not be displayed on the second day.

Warning - if there is no match in progress and you ask for 2 days the Play-Cricket server throws an error and you get a HTTP 500.

https://www.play-cricket.com/embed_widget/live_scorer_widgets?club_id=3370&days=2



There is a much better way of doing this and have the layout, theming, and display completely under your control.

The engine of the process is the call to

https://www.play-cricket.com/embed_widget/live_scorer_widgets?club_id=3370&days=1
 Where the numeric is your Play-Cricket site ID.


This returns a JSON structure similar to this

{
  "matches": [
    {
      "home_club_name": "Slough CC",
      "home_team_name": "1st XI",
      "home_team_id": "22898",
      "away_club_name": "High Wycombe CC",
      "away_team_name": "1st XI",
      "away_team_id": "33634",
      "match_date": "30/06/2018",
      "match_id": 3333170,
      "league_name": "Home Counties Premier Cricket League",
      "competition_name": "Division 1",
      "competition_type": "League",
      "batted_first": "",
      "toss_won_by_team_id": "",
      "home_team_score": "Yet to bat",
      "away_team_score": "Yet to bat",
      "fixture_id": 3333170
    },
 All you really need out of this is the club and team names and the competition and league name and the fixture ID.

One of the neatest ways of displaying interesting data which will change on an irregular basis is by using what is known as a carousel.

You'll Never Walk Alone 

The carousel displays a number of panels by performing a sliding transition between each panel of data. You will have seen these on many websites, most of them are highly irritating an unnecessary but are a really good fit for what we are trying to achieve.

To make life even easier using Bootstrap CSS we can create a carousel with a minimum of coding.

To get a feel for how this displays take a look at the following short video -

Live Score Carousel

Here we have a smallish screen element which will show only active matches and can be put on any page of your club website.

Real-time data?

Not even close. There are many steps involved in getting the scores from your laptop to Play-Cricket.

PCS Pro generates a lot of data, much of which is of no interest to Play-Cricket such as wagon wheels, pitch maps, temperature and pressure data and so on. If all of this was sent to the Play-Cricket server it would fall over. Your data is routed via an intermediary NV Play server which filters out all the data which has no slot in Play-Cricket, it then sends the data onward via the Play-Cricket scoring API. At any stage your data may get throttled, you may find you can only update at the end of an over or there are delays through the network and intermediary servers.

And finally, as with all Play-Cricket pages there is no asynchronous update capability so that to get the latest data you have to refresh the page. As anyone who has tried to follow the second innings on a mobile knows this is intensely annoying. You refresh the page and then have to expand the second batting card each time.

This is also true of the JSON webservice, to get the latest data you need to make another call to the webservice. The default Play-Cricket widget configuration performs a refresh every 3 minutes, you could lower that but it may cause problems with the P-C server if it is severely under load due to short timescale refreshes so probably best leave it at that.

For those who wish to experiment the sample code is shown below. If you want any help with this give me a shout.


<!DOCTYPE html>

<html>

<head>

<link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.7/css/bootstrap.min.css" integrity="sha384-BVYiiSIFeK1dGmJRAkycuHAHRg32OmUcww7on3RYdg4Va+PmSTsz/K68vbdEjh4u" crossorigin="anonymous">

<script src="https://code.jquery.com/jquery-3.3.1.min.js" integrity="sha256-FgpCb/KJQlLNfOu91ta32o/NMZxltwRo8QtmkMRdAu8=" crossorigin="anonymous"></script>

<script src="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.7/js/bootstrap.min.js" integrity="sha384-Tc5IQib027qvyjSMfHjOMaLkfuWVxZxUPnCJA7l2mCWNIpG9mGCD8wGNIcPD7Txa" crossorigin="anonymous"></script>

<title>Live Score Carousel</title>

<meta charset="utf-8" />

<style>

.blue-border {

border: 2px solid darkblue;


}
.orangebox-small {

font-size: 10pt;

text-align: center;

color: White;

background-color: rgb(252,176,59);

overflow: hidden;


}
</style>

</head>

<body>

<div class="container-fluid">

<div class="row">

<div class="col-xs-8 col-md-8" >

</div>

<div class="col-xs-4" id="livescores">

</div>

</div>

</div>

<script type="text/javascript">


$(document).ready(function () {

scoreswidget($('#livescores'));

});



var liveScores = function () {

$.ajax({

type: 'GET',

url: "https://www.play-cricket.com/embed_widget/live_scorer_widgets?club_id=3370",

dataType: "json",

success: function (data) {

$("#carscore").carousel("pause").removeData();

$('#lsinner').empty();

$firstscore = true;



for (i = 0; i < data.matches.length; i++) {

// display only matches with 'live scores'

if (data.matches[i].home_team_score != "") {

if ($firstscore) {

$html = '<div class="item active"><div >';

$firstscore = false;

}

else

$html = '<div class="item"><div >';

if (data.matches[i].competition_name != "") {

$html += '<div class="col-x-12 text-center orangebox-small" >' + data.matches[i].league_name + '</div>';

$html += '<div class="col-x-12 text-center orangebox-small">' + data.matches[i].competition_name + '</div>';

}

else // its a friendly

$html += '<div class="col-x-12 text-center orangebox-small">' + data.matches[i].competition_type + '</div>';

$html += '<div class="col-x-12 text-center">' + data.matches[i].home_club_name + ' ' + data.matches[i].home_team_name + '</div>';

$html += '<div class="col-x-12 text-center">' + data.matches[i].home_team_score + '</div>';

$html += '<div class="col-x-12 text-center">' + data.matches[i].away_club_name + ' ' + data.matches[i].away_team_name + '</div>';

$html += '<div class="col-x-12 text-center">' + data.matches[i].away_team_score + '</div>';

$html += '<div class="col-x-12 text-center"><a target="_blank" href="http://www.play-cricket.com/website/results/' +

data.matches[i].fixture_id + ' ">Scorecard</a></div>';



$html += '</div></div>';

$('#lsinner').append($html);

}

}

$("#carscore").carousel();

setTimeout(function () { liveScores(); }, 30000);

}

});

}









var scoreswidget = function (placeholder) {

placeholder.append(

'<div class="col-xs-12 blue-border" style="margin-top: 10px;">'

+ ' <ul id="tabs" class="nav nav-tabs">'

+ ' <li class="active"><a data-toggle="tab" href="#live">Live Scores</a></li>'

+ ' <li class=""><a data-toggle="tab" href="#live">Other News</a></li>'

+ ' </ul>'

+ ' <div class="tab-content" style="height: 120px">'

+ ' <div id="live" class="tab-pane fade in active " style="margin-top: 10px">'

+ ' <div id="carscore" data-interval="10000" class="carousel slide" data-ride="carousel"> '

+ ' <div id="lsinner" class="carousel-inner">'

+ ' </div>'

+ ' </div>'

+ ' </div>'

+ ' </div> <!-- tab content -->'



//sponsor logo if you wish

+ '<div class=" text-center darkBlue" style="margin-bottom: 10px; ">'

+ '<div class="midlinks"><a target="_blank" href="http://www.pdx.co.uk/"><img style="height: 90px;" src="/images/pdx.jpg" /></a>'

+ '</div> </div> </div>'

);





liveScores();



}
</script>




 </body>

</html>