I run some Bash scripts on my Synology to maintain my birdwatching camera. I'm having a weird problem with one of them, and the kind folks on this BBS have taught me so much about shell scripting in Linux before. I hope y'all can help here?
I've tried to pare down the problem to the simplest and clearest possible example, included in the code below.
Basic description:
The script uses curl to retrieve this web page:
http://www.google.com/search?q=sunrise - Google uses the calling browser's IP address to locate the user on the globe, then returns today's sunrise time at the top of the search results, in big bold text. The script parses that resulting web page for the first thing that looks like a time-of-day and prints it out. I actually have more complex code that retrieves both sunrise and sunset, but I've simplified this code to just look for sunrise, so I can show the example clearly.
This all used to work fine. But recently, Google changed their output so that instead of outputting a simple "7:05 AM" for example, they had to get fancy and change the space in the middle to a unicode nonbreaking space (code 0x202f). This made me change the parser a little bit so that it has to parse for any single character in between the digits and the AM/PM. The change to the Google results, and subsequently, the change to the parser, is when the trouble started.
The weird thing is that the parser works in every test attempt that I make by hand. Every attempt I make to debug the thing in an easy/simple way, it never fails. It only fails in this one situation that's hard to debug.
The problem:
- When I run the script at the Mac shell prompt, or when I SSH into the Synology and run it from the shell prompt there, it works perfectly.
- Only when I run the script as a task in the Synology Task Scheduler, only then, it fails.
- It fails in the weirdest way.
Failure details (when it fails):
- In the Task Scheduler setup, I pipe the script's output into a log file, to see what's going wrong. The "Run Command" is a user-defined script that looks like this:
bash "/volume1/homes/admin/CrowCam/TimeTest.sh" > "/volume1/homes/admin/CrowCam/TimeTest.log" 2>&1
- When I run that Task in the Synology Task Scheduler, it fails, and then I grab TimeTest.log and load it up in my Sublime Text editor.
- I see the log: The first grep statement in the script has returned nothing, a null string. (But only when running the script from the Synology Task Scheduler.)
- The string it's grepping into is fine. The entire web page is indeed printed there in the log, including the string it's grepping for, with 0x202f in the middle as expected.
- When I use Sublime's regex search feature, using the same regex search that the grep statement uses, it
succeeds and finds the thing that the grep couldn't find. So I know that the google part of the script is correctly returning the expected data, and that the data is greppable.
The weird part is that when I SSH into the Synology and run the same script unchanged (regardless of whether I "sudo" or I launch it with Bash or with Sh), it all always works perfectly, no failure.
Any ideas what could cause that Grep to fail?
Here's the test code:
#!/bin/bash
# ---------------------------------------------------------------------------
# TimeTest.sh - Retrieve the time of the sunrise from Google.
#
# Google use the IP address of the user to find the location and then prints
# the sunrise time on the screen for the user's location, at the top of its
# search results.
# ---------------------------------------------------------------------------
# The User Agent string is required in order for Google to return a result.
# Without the User Agent string, Google prints an error message.
userAgent="Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) \
Chrome/51.0.2704.103 Safari/537.36"
googleQueryUrl="http://www.google.com/search?q=sunrise"
echo ""
echo "Performing Google query: $googleQueryUrl"
googleQueryResult=
googleQueryResult=$( curl -L -A "$userAgent" -s "$googleQueryUrl" )
# Print the HTML page result to prove we got it (sorry it's so big).
echo ""
echo "Google Query Result: $googleQueryResult"
echo ""
# Sunrise time is the first thing in the result which looks like a time string.
# For example "7:05 AM" appears in the results near the top. However, the
# "7" might sometimes be two digits, and the space between the "05" and the "AM"
# is a unicode nonbreaking space, code 0x202f. So the regex below searches for
# one or more digits, a colon, two more digits, any single character such as the
# nonbreaking space or a regular space, then AM or PM.
timeWithWeirdSpaceInTheMiddle=$( echo $googleQueryResult | \
grep -o -m 1 '[0-9][0-9]*:[0-9][0-9].[AP]M' )
# Convert this back into a more usable time string by parsing out the time
# digits and the AM/PM, and then re-inserting a regular space between them.
firstTimeSection=$(echo $timeWithWeirdSpaceInTheMiddle | grep -o '[0-9][0-9]*:[0-9][0-9]')
secondTimeSection=$(echo $timeWithWeirdSpaceInTheMiddle | grep -o '[AP]M')
finalSunriseString="$firstTimeSection $secondTimeSection"
# Output all of the things that were retrieved from the parsing.
echo "timeWithWeirdSpaceInTheMiddle: $timeWithWeirdSpaceInTheMiddle"
echo "firstTimeSection: $firstTimeSection"
echo "secondTimeSection: $secondTimeSection"
# Exit with an error if either of the time sections came up empty.
if [ -z "$firstTimeSection" ] || [ -z "$secondTimeSection" ]
then
echo ""
echo "ERROR: Time was not correctly retrieved"
echo ""
exit 1
else
echo ""
echo "finalSunriseString: $finalSunriseString"
echo ""
exit 0
fi
Previous Topic
Index
