How to chart the progress of Government Petitions (using PHP and Excel/Libre Calc) ...

[webtrekker]

Ok, before I get down to details, here's what I regard as the issue that needed a solution.

 

The Gov Petitions site, while great for noting the current total of signatures received for any petition, lacks any way of plotting successive totals over a period of time to see the trend of the count. I Googled for sites that might provide this data but came up with nothing ................ so I made my own!

 

I started with the Petition for an Election at https://petition.parliament.uk/petitions/700143

 

I needed some way of scraping that signature count from the page at set time intervals and logging it. I found that the best way to access the signature count was via the JSON data provided by the Petition site itself, at https://petition.parliament.uk/petitions/700143.json

 

If you inspect this data you will find an attribute named 'signature_count.'

 

 

 

This is handy, as it enables us to extract the count, BUT, it only updates whenever the page is loaded. So, what is needs to be done is to write a PHP script (which resides on a server, not in the browser) that is called by a PHP cronjob at regular intervals. The PHP script will decode the JSON data and extract the signature_count and append it to a log file, which is set up as a comma-separated CSV file on the server in the same directory. A Timestamp is also added to each appended signature_count value.

 

The PHP script itself is simple ...

 

track_petition.php

<?php
// URL of the petition's JSON data
$json_url = "https://petition.parliament.uk/petitions/700143.json";

// Fetch the JSON data
$json_data = file_get_contents($json_url);

// Decode the JSON data
$data = json_decode($json_data, true);

// Extract the signature count
$signature_count = $data['data']['attributes']['signature_count'];

// Prepare the CSV file
$csv_file = 'el_count.csv';

// Check if the file exists
$file_exists = file_exists($csv_file);

// Open the CSV file for appending
$file = fopen($csv_file, 'a');

// Write headers if the file is new
if (!$file_exists) {
    fputcsv($file, ['Timestamp', 'Signature Count']);
}

// Append the signature count with a timestamp
fputcsv($file, [date('Y-m-d H:i:s'), $signature_count]);

// Close the file
fclose($file);

echo "Signature count of $signature_count has been appended to $csv_file.";
?>

 

 

To make the whole process automatic, you can make up your own cronjob, or use a website (as shown below) to do the job for you with ease ...

 

 

This runs every 10 minutes, 24/7, and runs the file track_petition.php which resides on your server. Each time it runs, it retrieves the signature_count JSON value and appends it to the file 'el_count.csv' along with a Timestamp. The result looks a bit like this ...

 

el_count.csv

 

 

Finally, it's simply a matter of taking the 'el_count.csv' data file and importing it to a spreadsheet (Excel, Libre Calc, etc) to produce a pretty chart, or whatever, which will show any trends in the data.

 

This chart shows a log of the Election Petition data over the time period of roughly 3pm to 10:30pm today, every 10 mins.

 

 

I added another column (C) to the spreadsheet to calculate the number of signatures received during each 10 min interval, independent of the Total. As you can see, it shows around 6,000 signatures being received every ten minutes! This is one Hell of a Petition!!!

 

Further thoughts ...

 

I could carry on adding more code to produce the graph automatically, without the need for a spreadsheet.

 

The routine could very easily be adapted to track ANY Petitions on the site. I've already done one to track the progress of a WASPI Womens Compensation Petition for my wife.

 

Enjoy. I hope the code may be of use to someone. Historical data is always great to have.

 

 

 

[webtrekker]

BTW, I only wish I had thought of this at the beginning of the Petition. I would have loved to have seen the chart in the initial stages as it rocketed skywards!

[Blencathra]

9 hours ago, webtrekker said:

Ok, before I get down to details, here's what I regard as the issue that needed a solution.

 

The Gov Petitions site, while great for noting the current total of signatures received for any petition, lacks any way of plotting successive totals over a period of time to see the trend of the count. I Googled for sites that might provide this data but came up with nothing ................ so I made my own!

 

I started with the Petition for an Election at https://petition.parliament.uk/petitions/700143

 

I needed some way of scraping that signature count from the page at set time intervals and logging it. I found that the best way to access the signature count was via the JSON data provided by the Petition site itself, at https://petition.parliament.uk/petitions/700143.json

 

If you inspect this data you will find an attribute named 'signature_count.'

 

 

 

This is handy, as it enables us to extract the count, BUT, it only updates whenever the page is loaded. So, what is needs to be done is to write a PHP script (which resides on a server, not in the browser) that is called by a PHP cronjob at regular intervals. The PHP script will decode the JSON data and extract the signature_count and append it to a log file, which is set up as a comma-separated CSV file on the server in the same directory. A Timestamp is also added to each appended signature_count value.

 

The PHP script itself is simple ...

 

track_petition.php

<?php
// URL of the petition's JSON data
$json_url = "https://petition.parliament.uk/petitions/700143.json";

// Fetch the JSON data
$json_data = file_get_contents($json_url);

// Decode the JSON data
$data = json_decode($json_data, true);

// Extract the signature count
$signature_count = $data['data']['attributes']['signature_count'];

// Prepare the CSV file
$csv_file = 'el_count.csv';

// Check if the file exists
$file_exists = file_exists($csv_file);

// Open the CSV file for appending
$file = fopen($csv_file, 'a');

// Write headers if the file is new
if (!$file_exists) {
    fputcsv($file, ['Timestamp', 'Signature Count']);
}

// Append the signature count with a timestamp
fputcsv($file, [date('Y-m-d H:i:s'), $signature_count]);

// Close the file
fclose($file);

echo "Signature count of $signature_count has been appended to $csv_file.";
?>

 

 

To make the whole process automatic, you can make up your own cronjob, or use a website (as shown below) to do the job for you with ease ...

 

 

This runs every 10 minutes, 24/7, and runs the file track_petition.php which resides on your server. Each time it runs, it retrieves the signature_count JSON value and appends it to the file 'el_count.csv' along with a Timestamp. The result looks a bit like this ...

 

el_count.csv

 

 

Finally, it's simply a matter of taking the 'el_count.csv' data file and importing it to a spreadsheet (Excel, Libre Calc, etc) to produce a pretty chart, or whatever, which will show any trends in the data.

 

This chart shows a log of the Election Petition data over the time period of roughly 3pm to 10:30pm today, every 10 mins.

 

 

I added another column (C) to the spreadsheet to calculate the number of signatures received during each 10 min interval, independent of the Total. As you can see, it shows around 6,000 signatures being received every ten minutes! This is one Hell of a Petition!!!

 

Further thoughts ...

 

I could carry on adding more code to produce the graph automatically, without the need for a spreadsheet.

 

The routine could very easily be adapted to track ANY Petitions on the site. I've already done one to track the progress of a WASPI Womens Compensation Petition for my wife.

 

Enjoy. I hope the code may be of use to someone. Historical data is always great to have.

 

 

 

This is very clever. I would keep the spreadsheet (as a separate data resource, just in case) alongside the graph. 

 

Thanks for sharing.

 

N.B. Best of luck with the WASPI Petition.

        Have signed!

Edited November 26, 2024 by Blencathra

[webtrekker]

Small update ...

 

I have added a routine to draw the graph as a PNG image in the browser, rather than having to muck around with a spreadsheet each time.

 

I have also coded a script to produce a graph of the WASPI Women Compensation Petition.

 

 

Call for a new Election Petition (showing data from 3pm Tuesday for 32 hours

 

 

WASPI Women Compensation Petition (showing data for 24 hours from midnight Tuesday)

 

You can easily see from both charts that activity dies off and nearly plateaus during the early hours but picks up throughout the day. Also, the Election Petition seems to be losing a bit of steam now as the line climbs at a lesser angle, but still looking good.

[webtrekker]

Wow! This one seems to have taken off in the last hour ...

 

Petition: Introduce a compensation scheme for WASPI women

We call on the Government to fairly compensate WASPI women affected by the increases to their State Pension age and the associated failings in DWP communications.

 

 

 

 

Edited November 27, 2024 by webtrekker

[webtrekker]

Further to my last post ...

 

I see the WASPI Women's Petition has made a nice little upturn this morning to achieve the 100,000 signatures status ...  

 

[webtrekker]

I've been working on an improved charting program that is responsive, zoom-able and pan-able, with Tooltips appearing when hovering over a data point.

 

Here's the WASPI chart as an example, showing the latest data as of today ...

 

 

Here's a zoomed-in portion with a Tooltip being displayed showing the Date, Time, and Signature Count for that point ...

 

 

Now that I've got that bit working fine, I'm going to concentrate on a more generalised program to track many popular and (IMHO) important Petitions from the Government website. When I'm done, I'll post a link so that any of you can access my charting app if needed. Being able to track a Petition over time is more useful than just finding the current Signature Count, and is a feature not offered by the Petitions site itself.

 

 

 

[Grumpy Owl]

One flaw I have found in the UK Petitions website is that it seems anyone can sign a petition, even if they are not in the UK.

 

Now I accept there is always the possibility that UK citizens may be working or residing elsewhere around the world at any given time.

 

But it seems to me that the petitions site could be open to abuse from bots or others that are prompted to sign such petitions.