GroveStreams Raspberry Pi - Monitoring Pi Metrics
This tutorial demonstrates how to upload Raspberry Pi system metrics along with predefined low disk and low memory events with notifications into GroveStreams.The tutorial walks through the creation of a GroveStreams user account, an organization, and the installation of a Python script on a Raspberry Pi that uploads the Raspberry Pi's metrics, such as CPU and memory usage, once every minute by using crontab.
The Python script can be easily changed or appended in order to upload additional metrics such as data collected by sensors that are attached to the Raspberry Pi.
This tutorial uses a Raspberry Pi - Model B (512 MB / Revision 2) with a 8GB SD Card. The operating system is Raspbian Debian Wheezy. The tutorial uses Python version 3.2 which is part of the Raspbian Wheezy image.
The number of streams created by this tutorial falls within the free monthly amounts of GroveStreams. Setting the sample frequency to one minute, as shown by this tutorial, will exceed the free transaction amounts after a few days.
Step 1: Create a Free User Account and Organization
Open a browser and navigate to the GroveStreams registration page and sign up.
After creating a free user account, log into GroveStreams.
You will be prompted to create an organization. Select Yes.
What's an Organization?
An organization is a Workspace, typically representing a home, business, or organization. Each organization has its own set of components, streams, dashboards, maps, and other items. An organization allows you to control user and device access. You are automatically the "owner" and given full access rights when you create an organization. Other users may invite you to their organizations with granted. All of the organizations, that you "own" or are a member of, will appear in your GroveStreams start page (the first page that appears when you sign in).
An organization is a Workspace, typically representing a home, business, or organization. Each organization has its own set of components, streams, dashboards, maps, and other items. An organization allows you to control user and device access. You are automatically the "owner" and given full access rights when you create an organization. Other users may invite you to their organizations with granted. All of the organizations, that you "own" or are a member of, will appear in your GroveStreams start page (the first page that appears when you sign in).
1. Enter a name for the new organization. It can be any name.
2. Expand the Advanced section.
3. Ensure Create with a system blueprint is selected.
4. Select the Raspberry Pi Metrics Blueprint system blueprint. This will create your organization with commonly used items such as units.
5. Click Create Organization.
1. Enter your newly created organization by clicking on its name.
Step 2: Install and Configure the GroveStreams Python Scripts on the Raspberry Pi
Install:
Manually
Click gs_pi_metrics.tar.gz to download the GroveStreams Raspberry Pi Python files. Uncompress gs_pi_metrics.tar.gz after the download has completed. This tutorial will assume the files are uncompressed within the user's "pi" home directory (/home/pi).
Or by Command Line
1. Open a command window on the Pi (ssh into your Pi or open a terminal window).
2. Navigate to the directory you want to install install the Python files to. For this example we will use the user's "pi" home directly: $cd /home/pi
3. Download the compressed files: $wget https://www.grovestreams.com/downloads/rpi/gs_pi_metrics.tar.gz
4. Uncompress the files: $tar xvzf gs_pi_metrics.tar.gz
This directory should exist after uncompressing the files: /home/pi/gs_pi_metrics
Click gs_pi_metrics.tar.gz to download the GroveStreams Raspberry Pi Python files. Uncompress gs_pi_metrics.tar.gz after the download has completed. This tutorial will assume the files are uncompressed within the user's "pi" home directory (/home/pi).
Or by Command Line
1. Open a command window on the Pi (ssh into your Pi or open a terminal window).
2. Navigate to the directory you want to install install the Python files to. For this example we will use the user's "pi" home directly: $cd /home/pi
3. Download the compressed files: $wget https://www.grovestreams.com/downloads/rpi/gs_pi_metrics.tar.gz
4. Uncompress the files: $tar xvzf gs_pi_metrics.tar.gz
This directory should exist after uncompressing the files: /home/pi/gs_pi_metrics
Configure:
The next step is to give the Raspberry Pi rights to upload into
your GroveStreams organization. This is done by having the
Raspberry Pi use a secret API key that belongs to your GroveStreams
organization.
The apiKey Python variable within the GroveStreams Settings section of the Python script file, /home/pi/gs_pi_metrics/GroveStreams.py, must be assigned the secret API Key:
1. Edit the file /home/pi/gs_pi_metrics/GroveStreams.py and set the apiKey variable in your Python file to match your GroveStreams organization Secret API Key. Use a file editor such as Nano: $nano /home/pi/gs_pi_metrics/GroveStreams.py
To find your Secret API Key:
1. Click the API Keys button
2. Select the Feed Put API Key (with auto-registration rights) key
3. Click View Secret Key
4. Select and copy the API Secret Key to the clipboard (Ctrl-c) and paste it into your sketch (Ctrl-v or Ctrl-Shift-v if using the nano editor) replacing the existing apiKey value.
The apiKey Python variable within the GroveStreams Settings section of the Python script file, /home/pi/gs_pi_metrics/GroveStreams.py, must be assigned the secret API Key:
1. Edit the file /home/pi/gs_pi_metrics/GroveStreams.py and set the apiKey variable in your Python file to match your GroveStreams organization Secret API Key. Use a file editor such as Nano: $nano /home/pi/gs_pi_metrics/GroveStreams.py
To find your Secret API Key:
1. Click the API Keys button
2. Select the Feed Put API Key (with auto-registration rights) key
3. Click View Secret Key
4. Select and copy the API Secret Key to the clipboard (Ctrl-c) and paste it into your sketch (Ctrl-v or Ctrl-Shift-v if using the nano editor) replacing the existing apiKey value.
Step 2: Run the GroveStreams Python Script on the Raspberry Pi
Run Once Manually:
Running the script manually is useful for diagnosing any issues
before scheduling the script. To run the script:
1. Open a command window on the Raspberry Pi
2. Run: $python3 /home/pi/gs_pi_metrics/GroveStreams.py
Look for any reported errors within the command window. Ensure your Raspberry Pi is registered within your GroveStreams organization.
Within your GroveStreams organization:
1. The notification button will indicate if you have any notifications. Some notifications report errors. Check for any notifications that may contain any upload errors.
2. Select the Components folder.
3. Click the Refresh button.
4. Your new component will appear under the Components folder.
5. Select a Stream.
6. Your latest measurements will appear within the Quick View panel. The Quick View panel will display No Data if no measurements have been uploaded.
Congratulations! You have successfully uploaded Raspberry Pi metrics into GroveStreams.
Double click a stream to explore or edit historical values. Click on the Dashboards tab to create dashboards and maps. Keep exploring and learning!
1. Open a command window on the Raspberry Pi
2. Run: $python3 /home/pi/gs_pi_metrics/GroveStreams.py
Look for any reported errors within the command window. Ensure your Raspberry Pi is registered within your GroveStreams organization.
Within your GroveStreams organization:
1. The notification button will indicate if you have any notifications. Some notifications report errors. Check for any notifications that may contain any upload errors.
2. Select the Components folder.
3. Click the Refresh button.
4. Your new component will appear under the Components folder.
5. Select a Stream.
6. Your latest measurements will appear within the Quick View panel. The Quick View panel will display No Data if no measurements have been uploaded.
Congratulations! You have successfully uploaded Raspberry Pi metrics into GroveStreams.
Double click a stream to explore or edit historical values. Click on the Dashboards tab to create dashboards and maps. Keep exploring and learning!
Schedule to Run Every Minute:
To configure a Linux crontab job to run the Python script every
minute perform these steps:
1. Open a command window on the Raspberry Pi
2. Run: $sudo crontab -e and add the following to the end:
*/1 * * * * python3 /home/pi/gs_pi_metrics/GroveStreams.py > /home/pi/gs_pi_metrics/out.txt
3. Save and apply crontab changes: Ctrl-X, Y, Enter
Congratulations! You have successfully configured your Raspberry Pi to upload its metrics once every minute.
1. Open a command window on the Raspberry Pi
2. Run: $sudo crontab -e and add the following to the end:
*/1 * * * * python3 /home/pi/gs_pi_metrics/GroveStreams.py > /home/pi/gs_pi_metrics/out.txt
3. Save and apply crontab changes: Ctrl-X, Y, Enter
Congratulations! You have successfully configured your Raspberry Pi to upload its metrics once every minute.
How it Works
The Python script on the Raspberry Pi gathers the Raspberry Pi's metric data and uploads them into GroveStreams each time it runs. The metric data sent to GroveStreams, as JSON, must include several keys so that GroveStreams knows where to store the data being sent:- api_key: This is your secret API key and is sent as a HTTP cookie with every upload request. It tells GroveStreams which organization the data will be inserted into and ensures the caller has the proper rights to make the call.
- componentId: This is the ID of a component within your GroveStreams organization that contains the stream that the data will go into. This example uses the Raspberry Pi's IP address for the ID. Component IDs must be unique throughout an organization.
- streamId: This is the ID of the component stream to place the data into. Stream IDs only need to be unique within each component.
- compTmplId: This is the ID of the component template to use when creating a new component when one does not exist. This is an optional part of the JSON. A default component with default streams will be created if it were not included. When you selected the Raspberry Pi Metrics Blueprint during the creation of your organization, a component template with an ID "rPi" was automatically created and included predefined events, units, and more.
- components.default.name: This is the name for the new component. Other new component and stream defaults can be inlcuded. We chose to only set the component name for this example. The name is from the Raspberry Pi /etc/hostname file.
We've made this example simple by utilizing a component template that was already defined within an organization blueprint. Users can view or modify existing component definitions by right clicking on them and selecting Edit Component. Users can also modify or create their own component templates and organization blueprints.
Troubleshooting
Watch for errors within the Raspberry Pi command window while running the script manually. Check /home/pi/gs_pi_metrics/out.txt for any errors after configuring crontab.If the call made it to a GroveStreams server, but an error still occurred, a GroveStreams system notification will be created that contains the error. While in the GroveStreams organization's Observation Studio, click on the Notifications button (located in the top toolbar. It looks like an envelope) and select the System tab. Click Refresh to list new notifications.
Ensure your Raspberry Pi can reach the GroveStreams servers. Do a command line ping to grovestreams.com from your Raspberry Pi. If that is unsuccessful, then try pinging the GroveStreams IP address directly (173.236.12.163). If you can ping the IP address, but cannot ping grovestreams.com, then you are experiencing a DNS lookup issue.
Useful Crontab Commands:
- List all crontab jobs: $sudo crontab -l
- Ensure crontab is running: $service cron status
- Check the system log for crontab messages: $grep CRON /var/log/syslog
Troubleshooting Checklist
- Did you remember to change the Python script file apiKey variable?
- Have you checked your GS organization for any notifications?
- Is the Python script reporting errors?
- Have you connected your Raspberry Pi to a network that has access to the world wide web?
- Still can't figure out what's wrong? Delete your GroveStreams organization, start over, and follow the instructions carefully. Sometimes it's easier and faster to start over than to try and figure out what step was missed or performed incorrectly.
Still having problems or you want to modify the solution and are not sure what to do?
Send us an email and we'll do our best to help you out: support@grovestreams.comHelpful Links
GroveStreams Help CenterGroveStreams Advanced Feed PUT API
GroveStreams Forum
Raspberry Pi Forums
