Category Archives: Project diary

Blog about small electronic or digital projects that I am undertaking

Formatting output in Python 3

I’ve had to work hard to get ‘pretty’ output from Python: it seems more complicated than with 50-year-old Fortran IV! Fortunately, Python’s formatting procedures have advanced, no doubt trying to resolve some of the issues.  Unfortunately, there are now many ways to do things, and if you search online, each website shows yet another method.

After considerable frustration, I found that Python has a full set of consistent, flexible and simple formatting codes – not a million miles away from those in Fortran!

An example

I want to show clock times in the conventional way such as 12:32 or as 05:09. Note the need for leading zeroes. The moon clock keeps the hours and minutes of the time in separate variables, which are already converted to integers.

To convert an integer value into a string formatted as above, the statement is:
hourString = ‘{0:02}’.format(hourInteger)
If hourInteger = 4, this would produce an output string  04

The format parameter ‘{0:02}’ says ‘convert the numerical value hourInteger into a string representation within a field length of 2 and pad the field with leading zeroes.

Converting a floating point value into a string, might use
clockphaseString = ‘{0:.3f}’.format(clockphase)
This could produce an output like 0.683.  The floating point value is correctly rounded to three decimal places.

clockazString = ‘{0:.0f}’.format(clockaz)
could produce an output like 306 (note that although clockaz is a floating-point value, it is displayed as a whole number).

You can format and insert any number of values from the format list: the ‘0’ at the beginning refers to the position in the value to be inserted.  Supposing the hour is 4 am stored as an integer in variable clockhour and the minute is 9 minutes past the hour, stored in variable clockmin, the following would create the character string 04:09

clocktime= ‘{1:02}:{0:02}’.format(clockmin, clockhour}

Note how the index value (the 1 in {1:02}) is used to correctly reposition the minutes and hours.

I don’t want to put all the detail into this blog, but the system is described well in this tutorial:
How to use string formatters in Python 3

I can recommend this method of formatting as being a powerful way of formatting numbers and inserting variable values into strings.

Problems with the clock

My software upgrade to include the graphical output window went smoothly enough, but I’ve started getting trouble with the clock mechanism. The first thing I noticed was that the manual advance button wasn’t working.  Obviously, stopping and starting the clock during numerous tests meant that it was slightly behind time and needed to be advanced manually.  I knew the GPIO was working because the moon lit up and the hands were advancing.  The most likely problem was a bad connection somewhere.

I checked all the connections and although some of the screws could be slightly tightened, this didn’t really seem to be enough to cause a problem.  Anyway, this got the button working again, but now both the clock motors are losing some pulses.  I have to wonder if the connectors on the Pi’s GPIO are not making proper contact.  It’s possible that in disturbing them, they have moved slightly as they are only connected by spring force.  Maybe there is the tiniest bit of corrosion causing the problem.  Otherwise, perhaps there is a problem with a component somewhere, although that doesn’t seem so likely.  Maybe I should buy a ribbon connector that would grip tightly.

The graphical interface

I have noticed that the graphical interface causes the Pi to run hotter – I am not sure to what extent this is caused when I RDP into the desktop, but it is running at over 50 % utilisation when updating, although it does drop to 0 % when idle.  When updating, the CPU temperature rises to 66 degrees, cooling to 56 degrees during idle periods.  This cycling range of 10 degrees each minute does seem rather undesirable.  Heat cycling can cause problems with connections and components. I could increase the interrupt timing to once every two minutes, but this might result in even larger temperature excursions.

Update on the temperature issue – 24th November 2018
Task Manager showed an increasing utilisation percentage, causing the temperature rise, and in fact there seemed to be two instances of Python3 running.  I started to suspect some sort of recursion, because it was taking ever longer for each update to complete, with long waits during the redraw process.  I wondered if this was due to multiple drawing to the same window, so I decided that instead of blanking the window with a coloured rectangle, which I thought would be quick and not cause a problem if I moved it, instead I would close the window and reopen it.  In trying this out, I realised that I was creating the window in the main program, but redrawing it in a subroutine, but I had not declared the instance of the window to be global.  It seemed that Graphics.py created another instance in the subroutine and perhaps it was creating a new instance each time I wrote to the window, although using some settings, perhaps that it was keeping local to itself, but this was clearly causing it extra work.  I’ve now made all the window and text object declarations global. Then I discovered the ‘undraw’ function to remove the old text before entering the updated text. (If you don’t remove the old text, it remains on screen, causing the text to ‘smear’.) The redraw now takes a fraction of a second, the utilisation has dropped to a typical value of 0 %, rising to 4 % momentarily during redraw. The temperature has stabilised at 47 degrees: result!

I should mention that if you open extra windows on the screen, Graphics.py tries to find somewhere to put the window: it tries to put it in the last place you moved it to, even though it is closed and redrawn. (I’ve realised that this must be handled by LXDE.)  Given the much reduced load on the processor, I was hoping that the power supply voltage would be higher and the clock movement would be more reliable, but this doesn’t seem to be the case, so more investigation is still needed.

Starting the moon-clock from the GUI

In the previous post, I explained how I was using a graphics window to display the current clock settings, and I showed how it was possible to generate a nice-looking output using a package called graphics.py by John Zelle.  I use xrdp (remote desktop protocol) to connect to the Pi from my PC, although I’m aware that there are other ways like using a terminal emulator.  However, I like to be able to use a familiar window-type interface with the file management, editing and web-browsing that it offers.

It turns out that this raises several issues.  Firstly, the graphical LXDE desktop on the Pi is only initiated after logon, not at bootup.  Therefore, you can’t write to it in during boot-up.  If you try, you get various weird errors like: Invalid MIT-MAGIC-COOKIE-1.  This is something to do with incorrect setting of the .Xauthority environment variable, which is apparently set for each user, and during bootup there isn’t a user, except root.  I have seen various suggestions for fixing this.  But if you get past that hurdle, you will then be told that it can’t find display :10 or something similar. Since LXDE hasn’t yet been started, that is to be expected.

Now, you can get programs to open automatically when the desktop loads by editing the file using
sudo nano /home/pi/.config/lxsession/LXDE-pi/autostart

You simply add the file that you want to run, as follows:

@lxpanel --profile LXDE-pi
@pcmanfm --desktop --profile LXDE-pi
@xscreensaver -no-splash
@point-rpi
@python3 /home/pi/Downloads/moon-clock-with-reporter18-11-09.py

In my case, I’ve added the last line starting @python3, which executes the moon–clock software.  Now experienced users will be aware that because the moonclock software uses the GPIO pins for both input and output, I should run it as super-user, using
sudo python3, otherwise I can’t access the GPIO.  But, if I run it as superuser, then I have the problem of lacking permissions to access the desktop (I get the can’t find display :10 error).   So, how do you fix this?  I thought that there has to be a way giving an ordinary user access to the GPIO.  It turn out that you can edit our old friend /etc/rc.local to do this by changing permissions as follows:

chgrp -R dialout /sys/class/gpio
chmod -R g+rw /sys/class/gpio

I admit that I’m not entirely sure what this does, but I understand that it gives any user who is a member of the dialout group (which includes all ordinary users) to read and write to GPIO, so you do not need to be a superuser.

And it works!  I boot the Pi into the desktop and then autostart the moonclock from the desktop, and it writes into a nice graphical window.  Now I always know that the clock should be showing.

GUIs

Well, I never thought I’d be blogging about GUIs (Graphical User Interfaces), at least not in the context of my moon clock.  After all, a clock is the original and best GUI of them all!

But I want to be able to check that the clock hands are showing the times that the software thinks they should be showing. The clock starts automatically in the background on power up and by default any output that it writes will disappear.  You can redirect it to stdout, which will display in a terminal window, or redirect it to a file, but this is a scrolling display that isn’t easy to read.

I would like the current settings to be displayed in a nice, clear, fixed window.

Python doesn’t have a ‘windows’ display built in but there is a big choice of GUI libraries that give a graphical interface.  The most popular seems to be tkinter, which is a Python wrapper for a GUI called Tcl.  Like most GUIs, tkinter is event-driven.  It runs in an infinite loop, and when the user does something, such as clicking a button, this creates an ‘event’. This causes tkinter to transfer control to the relevant function to deal with the button click, and when this has completed, it then returns to the infinite loop until another event occurs.  This makes sense for most apps, as it means that control always returns to the user via the GUI.

I knocked together a few lines of code using Tkinter, but found that it would not display unless I called the infinite loop or the update function, and once in the loop, execution of the code would not proceed further.  Basically, it wanted user input.  Since the clock is driven by timer events rather than an external user, this might not matter – as long as tkinter passed control to the timer event – but on the other hand it might not work.  (The documentation doesn’t seem to address this point.)   Even if it could work, it is quite involved and would need some changes to my own waiting loop.

I searched around for a GUI that is not event driven and came across easyGUI. This uses the Tk/Tcl drivers but is not event-driven.  Each GUI feature is a separate function. However, after displaying the window relevant to the function, again it waits for user input, which it will never get, so it’s no better than Tkinter.

Suddenly, the light dawned.   My question about getting a simple text output in a window has been asked on a lot of forums, and all the advice is to use a GUI.  But this is wrong!  You only need a graphics routine that can draw on the screen in a window.  I searched around for something very simple, and came across csc161graphics. This uses a library called graphics.py written by John Zelle.  It is said to be a wrapper for Tkinter!  But it is for creating things on-screen, not a GUI.   I downloaded and installed it using the following command.
sudo pip3 install –upgrade http://bit.ly/csc161graphics

csc161graphics is actually a computer course in the USA, so it has helpful tutorial material.  I only want to create a Text Box with some writing in it, that will stay on-screen, and can be updated.

This was very easy to do.  After importing the graphics library, you call a function to open the window as follows:

from graphics import *
# Create window with title and dimensions
win = GraphWin('Moon Clock', 600, 200) 
# Set the background colour fill of the window
win.setBackground('turquoise4')
# Compose the text to display
label = Text(Point(300,100),'The clock is now up and running')
# Set the text colour
label.setTextColor('yellow')
# Set the text size
label.setSize(12)
# Write the text into the window
label.draw(win)

I’m only showing some dummy text ‘The clock is now up and running’. The hardest bit was to format the real numerical information that I want to show in the window.  The spacing still isn’t perfect – it is difficult to get this right with a proportional-spaced font –  but you will notice that the numbers are displayed in the format you’d expect. I would have to cover that in another post.

Of course, now I have text display using a graphics interface, it is tempting to consider making a replica graphical clock display too!

The change from BST to GMT

How did the moon clock cope with this change – the first occurrence since we started?  I think the answer is 80 %.  Firstly, it did correctly recognise that BST had ended and this meant that npulse (the number of pulses needed at the end of each minute) was negative until the current time caught up with the displayed time.  So the s hand waited for one hour. (I did not get up to watch this).  In the morning, the moon was illuminated (it is near full moon with the moon up for 15 hours) and the s hand showed the right time. So far so good.

But the next test was whether the new rise and set times could be correctly displayed.  The problem is that on the 27/28th October, the rise time was 19:45 and the set time was 10:51, whereas on the 28/29th October, the rise time was 19:30 and the set time was 11:56. So the rise time was 15 minutes earlier on 28th than the previous day. This was caused by clocks going back an hour. Since the clock can’t go backwards, it would have to go forwards about 23 hours and 45 minutes, needing 43,000 pulses.  The algorithm decides that it is better to wait for 24 hours than to pulse this far, so it flashes the moon 500 times as an indicator.  Unfortunately, the counter in the flashmoon function  had somehow got deleted, so it was stuck in an infinite loop!  This seems like a silly error when the hard part was working, but it shows how easy it is to make a mistake when editing.  So I had to correct the error in the flashmoon function, adjust the rise and set times by hand and restart the clock.

Of course it will be a year until this situation arises again – when summer time comes into effect, times go forward and that’s not a problem.

Improvements to the Moon Clock Software

The moon clock seems to be keeping the correct time and to advance to the new rise and set times correctly.  So what improvements needed?

The main issue is that when running headless, there is no text output to tell you what the clock should be showing. This is a nuisance if you need to set the clock hands.  It is of course possible to direct the standard output to the console, but I have not been able to determine what buffer size is allocated, or how to flush it.  It could fill up after months or years. The same issue could apply to a file.  Moreover, directing to a file isn’t good because the output is buffered and only written when the buffer is full. Given the small amount of output from the clock, you’d never know what the current readings are.

It would be better to output to a GUI box nicely formatted to show just the expected information.  It is possible to do this in Python and my next step will be to try this out.

There is another small change I’d like to make.  We know that we are limited on the accuaracy that the h hand can show the set time, but there shouldn’t be much of an issue with the rise time.  Except that the rise time can be 15 hours or more after the previous set time. The present algorithm calculates the rise time display to be accurate at the time it is evaluated, 1.5 hours after the previous set time.  So the s hand can have moved by 13 or 14 hours before the rise time is reached, during which the m hand has moved about 13 or 14 minutes.  By allowing for this, we can get a more accurate rise time.  It can’t be completely accurate, of course, because the hands are all geared together.

 

The Moon Clock Software Published at last

The Moon Clock  has been soak-tested for a month now and seems to be running as designed.  So you can download it from the link below.

moon-clock-with-azimuth positioner18-08-18

Note that it has been given a file type of .txt (text).  If it is stored as a .py file, which is an executable file type, spam filters will  prevent it from downloading, as a precaution in case it contains malicious code.

As it is, you can review the code NotePad, or better in any code editor, such as NotePad++, which will colour the code to help you to follow its structure.

You will need to change the file type to .py in order for NotePad ++ or the IDLE interface in the Raspberry Pi, to be able to recognise it as Python code.  You can’t run it until you change the file type.

The code is fully commented and is fully described in my blog and on the Moon Clock Software page.  I hope it will be of interest and look forward to any (positive) feedback.

Final testing

I’ve been tidying up the moon clock operating software, removing redundant bits and clarifying the comments.  In the process, I’ve found some minor bugs, but who knows if I’ve introduced some new ones?  So I have decided to let it run for a while longer before releasing it.

Overall it seems to be running as intended.  The new moon is in two days time.  The clock is showing very little of the moon is visible: I wonder if it will last out two more days?  It will be a crucial test of the maths when the set time passes midnight. This will not be for another 11 days.  If this works correctly, we should be good to release the software fully.

I have previously said the clock motor was still dropping the occasional ‘tick’.  I decided to lengthen the ‘off’ time of the pulse from 17 to 20 ms. It looks as thought this might be enough to allow the rotor to settle properly.   I’ll need a 2-hour shift to give it a really good test of 3600 or more pulses, but over the last few days, the set time has only changed by a few minutes, so I’ll need to wait a bit longer.

Total Lunar Eclipse on Friday

Excitement in the Moon Clock World! There will be a total Lunar Eclipse on Friday 27 July 2018, visible from most of the UK, including London. 

Friday is the day of the Full Moon: lunar eclipses occur when the moon and sun are on the opposite sides of the earth, which is also the condition for Full Moon. However, the earth’s shadow must fall completely over the moon for a total eclipse. Because the moon’s orbit is slightly tilted relative to the earth’s orbit round the sun, it is only occasionally that the moon crosses the directly into the earth’s shadow at full moon. Usually, it is slightly above or below the shadow.  The eclipse will actually start before the moon has risen in London.  The eclipse will be total at 20:49 (moonrise) reaching a maximum at 21:21 and totality will end at 22:13.  During this period it will be very close to the horizon, so will only be visible from locations with a clear view to the South East.

The next total lunar eclipse visible in London  is on 21 January 2019.  This will occur during the small hours, starting at 2:36 am.  There will be a partial lunar eclipse in July 2019.

Lunar eclipses never occur alone:  there is always a solar eclipse at the new moon preceding or following the lunar eclipse, when the moon comes directly between the sun and the earth. Sometimes, there can be solar eclipses at both these new moons, and that is what is happening this summer!  Of course,  solar eclipses may not be total, and may not be visible across the whole globe.  Indeed, due to the relatively small amount of obscuration caused by the moon, total solar eclipses are much rarer than lunar eclipses: they do not last as long and are visible from a much smaller area – sometimes visible only over areas of ocean.

Apparently, there was a partial solar eclipse  on 13th July 2018 and there will be a partial solar eclipse on 11 August, but it will not be visible in London.   The next fairly full solar eclipse in London is not until the evening of 12th August 2026.

I gather that the ephem library is capable of calculating eclipses, but I shall not be adding this to the clock!  However, if I get a photo of the Lunar Eclipse, I will put it on  here.

Update
After more than a month of very hot, dry weather, guess what?  At 6 pm on the evening of the eclipse, there was a deluge!  No thunder, no lightning, just torrential rain for half and hour.  The sky had been clouding over since mid-afternoon, and I hoped that after this storm, it would clear.  It didn’t.  In hope, rather than in expectation, I headed for the North Downs above Reigate.  From here there was a clear view to the south, with the south downs, running along the channel coast, in the very far distance.  But in the far south, the sky was heavy.  And I could see flashes of lightning in those clouds.  An occasional spectacular fork that spanned tens of miles.  The distant storm moved off eastwards, so at least I stayed dry, but there was no chance of seeing the moon.  So I had to find this photo.

That’s been my luck with eclipses. I went to Cornwall for the August 1999 total solar eclipse, but I encountered the thickest cloud cover imaginable and it didn’t even get particularly dark at totality.  Is there something about the position of the sun and moon that causes the jet stream to deviate and drag wet weather in from the Atlantic?

The next lunar total eclipse is in January … not a good time for a clear sky, but we can live in hope!

Astronomical clocks

On Monday 15th July, I went ringing at Ottery St Mary in Devon. They are a lovely 18½ cwt eight cast by Taylor’s in 1949.  Glancing around the tower (a ground floor ring with a long draught) I saw what looked like a birdcage clock mechanism in a gallery some height above the floor.

Fifteenth century clock mechanism
Fifteenth century clock mechanism

The ringers pointed out that I had missed the most important part – a fantastic astronomical clock dial, which is visible from the crossing. (Shown at the top of this post)

I was told by the ringers that the clock mechanism was found in parts by a horologist in 1907 who put it all back together ‘with the bits he could find’. You can see from the photo that much of the gearing is very badly worn, although this is hardly unexpected in a 600-year-old mechanism that has been running 24 hours a day.  The clock has ticked  about 19 billion seconds! The ‘anchor’-style escapement looks newish to me and is probably part of the 1907 renovation. The angel on top of the dial case is not an original part of the clock – it was taken from the pulpit canopy when that was removed some long time in the past.

Sadly, only the ‘going’ train is working and that is now electrically-wound, since the church could not find anyone who was prepared to wind it daily. The clock is thought to date from about 1400, but it has a pendulum escapement, which would have been added later, since these were not invented until the mid-1600’s.  Although called an astronomical clock, this one shows the time on a 24-hour dial, and the ‘age’ of the moon in days.  The moon is indicated by a sphere which is set into the dial plate, and is painted half white and half black. It turns on a radial shaft to show the phase of the moon.   However, to me the most interesting thing is that the gearing for the dials is all within the dial case – the clock mechanism simply turns a drive shaft at some steady rate – I didn’t have time to study it closely.

The dial mechanism
The dial mechanism

The ringer who showed it to me explained that unfortunately the moon-age pointer and the sun-time pointer are simply on opposite ends of the same rod, so in fact the moon-age pointer is incorrect – it only shows the clock time.  He doesn’t know why it’s like that, but I suspect the gearing for the 29.5 day moon rotation has been lost. Or perhaps it was disconnected because it added too much friction for the going-train to cope. I wasn’t able to look at this closely owing to it being quite dark and the fact that I needed to return to base.  Sadly, the clock was not fully working, showing neither the correct time nor the correct moon phase (just past new moon) as some component had been removed for overhaul by the current horologist.

This clock is quite simple by the standards of other astronomical clocks, but it is undoubtedly one of the oldest surviving examples of a clock with a dial.  The clock at Salisbury Cathedral, for example, does not have a dial, but sounds the hours on a bell – the word ‘clock’ being, of course, derived from the French ‘cloche’, meaning a bell.

Nevertheless, it involves some very intricate precision work.  It is made entirely from wrought iron, but how did those early blacksmiths form the circular wheels and cut the teeth?

Of course, the original verge-and-foliot escapements were not very accurate, as the swing of the verge depended on the amount of force applied, which would vary due to changes of stiffness in the gear trains – often gaining or losing half an hour a day.  But better than relying on the sun in such a cloudy country as England.

By this token, I can call my moon clock an astronomical clock, but I think I’d need to design a fancy dial first!