Category Archives: Project diary

Blog about small electronic or digital projects that I am undertaking

Hardware and software for making demo videos

A new venture for me.  I need to make some videos to show how to use my professional software. This means that I need to be able to capture my computer screen and for me to do voice-overs.

You would think that Adobe Premiere Elements could do this, but no, you have to buy an expensive add-on (called, I think, ‘Captivate’) designed for people who are producing professional on-line teaching aids and courses. But it incorporates all sorts of features that I really don’t need, at least for now.

Screen capture software. I searched around for screen capture software and after quite a bit of research,  came across OBS Studio, which is a free screen capture and streaming program.  I’ve never thought that I would be into streaming, as I didn’t think it would ever be practicable for me, but perhaps that will change in the near future.

OBS Studio. I managed to download and install OBS Studio without any problems, and to record a test demonstration of my software, but I have had trouble in the past with decent audio, even with good microphones.

Microphones. In my professional work, an accurate microphone costs £1000 and that doesn’t include pre-amplifiers and other accessories, although the microphone will come with a calibration certificate and a flat frequency response of 6 to 20,000 Hz.  Of course this is completely over the top for voice work, where a range of 340 to 3400 Hz would be OK for clarity.

So I searched for ‘streaming’ microphones and of course there is a vast range.  In the old days, cheap microphones used a quartz piezo-electric crystal.  These have the advantage that they generate a high output voltage without needing a pre-amplifier, but they have an irregular frequency response.  Dynamic microphones are basically like loudspeakers in reverse – a diaphragm vibrates a coil between magnets, generating a voltage in the same way as a dynamo. They also produce a high* output voltage, but they are delicate – the voice-coil suspension can be damaged if they are dropped or knocked.  So condenser microphones have become popular.  Here the diaphragm is a thin sheet of metal or metallised plastic stretched over a frame and place very close to a metal back-plate.  An electric charge (a voltage) is put across the two and this is connected to a ‘charge’ amplifier which has a very high input impedance.  The two plates of the microphone act as a capacitor, or condenser in old terminology, and as the diaphragm vibrates, the capacitance changes accordingly, so that the voltage on the amplifier changes.  This is amplified to a value suitable for the recorder input.  At one time, the electronics were difficult and expensive. Ideally the amplifier needs to be close to the microphone, within the same housing.  I still see references to a ‘tube’ (i.e. a thermionic valve), used due to their high input impedances, but MOSFET transistors can have equally high input impedances, and I don’t see why anyone would use a ‘tube’ now.

*all things are relative – we are talking about millivolts!

To cut this long story short, I researched a suitable microphone.  I want something of adequate quality, and was totally disappointed by a lack of technical details on most of the products. Hardly any of the specs discuss the microphone technology, the frequency response or output level.  They all say silly things like ‘professional’, ‘high-definition audio’ (I still have no idea what that could mean).

USB Microphones. So I found two USB microphones that seemed possible candidates.  USB microphones are usually of the ‘MEMS’ (micro-electro-mechanical systems) type commonly used in mobile telephones.  Essentially, this is a condenser microphone fabricated on a silicon chip and directly connected to a special chip containing amplifiers and an analogue to digital converter.  These process the output into a digitised signal,   These can have excellent quality because the A/D converter can be optimised to the characteristics of the microphone cartridge and the required frequency response, and because the connecting cable carries a digital signal, it is fairly impervious to noise pickup. It doesn’t need to be impedance or voltage-matched to your sound card.  Only a few years ago, this would have made the microphone expensive, but nowadays specialised integrated circuits can be remarkably cheap.  They do require a power supply, but the USB connector can provide this.

My choice. The two I have bought – yes, I have ordered them and awaiting delivery – are KLIMTM Talk, which claims it is ideal for Skype, VOIP and other audio calls, and cost £24; and Blue Yeti, which claims it is for recording and streaming.  This is a condenser microphone with three capsules and an adjustable directivity pattern and gain control. It cost £85.  This does seem like a good specification – we will see.

My initial testing shows that both give excellent sound quality. The more expensive Blue Yeti gives a more ‘spacious’ sound but is more sensitive to ‘mouth’ sounds, whilst the KLIMTM has a more ‘dry’ (less spacious) sound and seems less affected by ‘mouth’ sounds.  It’s not possible to say at this stage which one I like best – they are both suitable for doing my voice-overs. I will make a more detailed review in my next post.

 

Formatting output in Python 3

I’ve had to work hard to get ‘pretty’ output from Python: it seems more complicated than with 50-year-old Fortran IV! Fortunately, Python’s formatting procedures have advanced, no doubt trying to resolve some of the issues.  Unfortunately, there are now many ways to do things, and if you search online, each website shows yet another method.

After considerable frustration, I found that Python has a full set of consistent, flexible and simple formatting codes – not a million miles away from those in Fortran!

An example

I want to show clock times in the conventional way such as 12:32 or as 05:09. Note the need for leading zeroes. The moon clock keeps the hours and minutes of the time in separate variables, which are already converted to integers.

To convert an integer value into a string formatted as above, the statement is:
hourString = ‘{0:02}’.format(hourInteger)
If hourInteger = 4, this would produce an output string  04

The format parameter ‘{0:02}’ says ‘convert the numerical value hourInteger into a string representation within a field length of 2 and pad the field with leading zeroes.

Converting a floating point value into a string, might use
clockphaseString = ‘{0:.3f}’.format(clockphase)
This could produce an output like 0.683.  The floating point value is correctly rounded to three decimal places.

clockazString = ‘{0:.0f}’.format(clockaz)
could produce an output like 306 (note that although clockaz is a floating-point value, it is displayed as a whole number).

You can format and insert any number of values from the format list: the ‘0’ at the beginning refers to the position in the value to be inserted.  Supposing the hour is 4 am stored as an integer in variable clockhour and the minute is 9 minutes past the hour, stored in variable clockmin, the following would create the character string 04:09

clocktime= ‘{1:02}:{0:02}’.format(clockmin, clockhour}

Note how the index value (the 1 in {1:02}) is used to correctly reposition the minutes and hours.

I don’t want to put all the detail into this blog, but the system is described well in this tutorial:
How to use string formatters in Python 3

I can recommend this method of formatting as being a powerful way of formatting numbers and inserting variable values into strings.

Problems with the clock

My software upgrade to include the graphical output window went smoothly enough, but I’ve started getting trouble with the clock mechanism. The first thing I noticed was that the manual advance button wasn’t working.  Obviously, stopping and starting the clock during numerous tests meant that it was slightly behind time and needed to be advanced manually.  I knew the GPIO was working because the moon lit up and the hands were advancing.  The most likely problem was a bad connection somewhere.

I checked all the connections and although some of the screws could be slightly tightened, this didn’t really seem to be enough to cause a problem.  Anyway, this got the button working again, but now both the clock motors are losing some pulses.  I have to wonder if the connectors on the Pi’s GPIO are not making proper contact.  It’s possible that in disturbing them, they have moved slightly as they are only connected by spring force.  Maybe there is the tiniest bit of corrosion causing the problem.  Otherwise, perhaps there is a problem with a component somewhere, although that doesn’t seem so likely.  Maybe I should buy a ribbon connector that would grip tightly.

The graphical interface

I have noticed that the graphical interface causes the Pi to run hotter – I am not sure to what extent this is caused when I RDP into the desktop, but it is running at over 50 % utilisation when updating, although it does drop to 0 % when idle.  When updating, the CPU temperature rises to 66 degrees, cooling to 56 degrees during idle periods.  This cycling range of 10 degrees each minute does seem rather undesirable.  Heat cycling can cause problems with connections and components. I could increase the interrupt timing to once every two minutes, but this might result in even larger temperature excursions.

Update on the temperature issue – 24th November 2018
Task Manager showed an increasing utilisation percentage, causing the temperature rise, and in fact there seemed to be two instances of Python3 running.  I started to suspect some sort of recursion, because it was taking ever longer for each update to complete, with long waits during the redraw process.  I wondered if this was due to multiple drawing to the same window, so I decided that instead of blanking the window with a coloured rectangle, which I thought would be quick and not cause a problem if I moved it, instead I would close the window and reopen it.  In trying this out, I realised that I was creating the window in the main program, but redrawing it in a subroutine, but I had not declared the instance of the window to be global.  It seemed that Graphics.py created another instance in the subroutine and perhaps it was creating a new instance each time I wrote to the window, although using some settings, perhaps that it was keeping local to itself, but this was clearly causing it extra work.  I’ve now made all the window and text object declarations global. Then I discovered the ‘undraw’ function to remove the old text before entering the updated text. (If you don’t remove the old text, it remains on screen, causing the text to ‘smear’.) The redraw now takes a fraction of a second, the utilisation has dropped to a typical value of 0 %, rising to 4 % momentarily during redraw. The temperature has stabilised at 47 degrees: result!

I should mention that if you open extra windows on the screen, Graphics.py tries to find somewhere to put the window: it tries to put it in the last place you moved it to, even though it is closed and redrawn. (I’ve realised that this must be handled by LXDE.)  Given the much reduced load on the processor, I was hoping that the power supply voltage would be higher and the clock movement would be more reliable, but this doesn’t seem to be the case, so more investigation is still needed.

Starting the moon-clock from the GUI

In the previous post, I explained how I was using a graphics window to display the current clock settings, and I showed how it was possible to generate a nice-looking output using a package called graphics.py by John Zelle.  I use xrdp (remote desktop protocol) to connect to the Pi from my PC, although I’m aware that there are other ways like using a terminal emulator.  However, I like to be able to use a familiar window-type interface with the file management, editing and web-browsing that it offers.

It turns out that this raises several issues.  Firstly, the graphical LXDE desktop on the Pi is only initiated after logon, not at bootup.  Therefore, you can’t write to it in during boot-up.  If you try, you get various weird errors like: Invalid MIT-MAGIC-COOKIE-1.  This is something to do with incorrect setting of the .Xauthority environment variable, which is apparently set for each user, and during bootup there isn’t a user, except root.  I have seen various suggestions for fixing this.  But if you get past that hurdle, you will then be told that it can’t find display :10 or something similar. Since LXDE hasn’t yet been started, that is to be expected.

Now, you can get programs to open automatically when the desktop loads by editing the file using
sudo nano /home/pi/.config/lxsession/LXDE-pi/autostart

You simply add the file that you want to run, as follows:

@lxpanel --profile LXDE-pi
@pcmanfm --desktop --profile LXDE-pi
@xscreensaver -no-splash
@point-rpi
@python3 /home/pi/Downloads/moon-clock-with-reporter18-11-09.py

In my case, I’ve added the last line starting @python3, which executes the moon–clock software.  Now experienced users will be aware that because the moonclock software uses the GPIO pins for both input and output, I should run it as super-user, using
sudo python3, otherwise I can’t access the GPIO.  But, if I run it as superuser, then I have the problem of lacking permissions to access the desktop (I get the can’t find display :10 error).   So, how do you fix this?  I thought that there has to be a way giving an ordinary user access to the GPIO.  It turn out that you can edit our old friend /etc/rc.local to do this by changing permissions as follows:

chgrp -R dialout /sys/class/gpio
chmod -R g+rw /sys/class/gpio

I admit that I’m not entirely sure what this does, but I understand that it gives any user who is a member of the dialout group (which includes all ordinary users) to read and write to GPIO, so you do not need to be a superuser.

And it works!  I boot the Pi into the desktop and then autostart the moonclock from the desktop, and it writes into a nice graphical window.  Now I always know that the clock should be showing.

GUIs

Well, I never thought I’d be blogging about GUIs (Graphical User Interfaces), at least not in the context of my moon clock.  After all, a clock is the original and best GUI of them all!

But I want to be able to check that the clock hands are showing the times that the software thinks they should be showing. The clock starts automatically in the background on power up and by default any output that it writes will disappear.  You can redirect it to stdout, which will display in a terminal window, or redirect it to a file, but this is a scrolling display that isn’t easy to read.

I would like the current settings to be displayed in a nice, clear, fixed window.

Python doesn’t have a ‘windows’ display built in but there is a big choice of GUI libraries that give a graphical interface.  The most popular seems to be tkinter, which is a Python wrapper for a GUI called Tcl.  Like most GUIs, tkinter is event-driven.  It runs in an infinite loop, and when the user does something, such as clicking a button, this creates an ‘event’. This causes tkinter to transfer control to the relevant function to deal with the button click, and when this has completed, it then returns to the infinite loop until another event occurs.  This makes sense for most apps, as it means that control always returns to the user via the GUI.

I knocked together a few lines of code using Tkinter, but found that it would not display unless I called the infinite loop or the update function, and once in the loop, execution of the code would not proceed further.  Basically, it wanted user input.  Since the clock is driven by timer events rather than an external user, this might not matter – as long as tkinter passed control to the timer event – but on the other hand it might not work.  (The documentation doesn’t seem to address this point.)   Even if it could work, it is quite involved and would need some changes to my own waiting loop.

I searched around for a GUI that is not event driven and came across easyGUI. This uses the Tk/Tcl drivers but is not event-driven.  Each GUI feature is a separate function. However, after displaying the window relevant to the function, again it waits for user input, which it will never get, so it’s no better than Tkinter.

Suddenly, the light dawned.   My question about getting a simple text output in a window has been asked on a lot of forums, and all the advice is to use a GUI.  But this is wrong!  You only need a graphics routine that can draw on the screen in a window.  I searched around for something very simple, and came across csc161graphics. This uses a library called graphics.py written by John Zelle.  It is said to be a wrapper for Tkinter!  But it is for creating things on-screen, not a GUI.   I downloaded and installed it using the following command.
sudo pip3 install –upgrade http://bit.ly/csc161graphics

csc161graphics is actually a computer course in the USA, so it has helpful tutorial material.  I only want to create a Text Box with some writing in it, that will stay on-screen, and can be updated.

This was very easy to do.  After importing the graphics library, you call a function to open the window as follows:

from graphics import *
# Create window with title and dimensions
win = GraphWin('Moon Clock', 600, 200) 
# Set the background colour fill of the window
win.setBackground('turquoise4')
# Compose the text to display
label = Text(Point(300,100),'The clock is now up and running')
# Set the text colour
label.setTextColor('yellow')
# Set the text size
label.setSize(12)
# Write the text into the window
label.draw(win)

I’m only showing some dummy text ‘The clock is now up and running’. The hardest bit was to format the real numerical information that I want to show in the window.  The spacing still isn’t perfect – it is difficult to get this right with a proportional-spaced font –  but you will notice that the numbers are displayed in the format you’d expect. I would have to cover that in another post.

Of course, now I have text display using a graphics interface, it is tempting to consider making a replica graphical clock display too!

The change from BST to GMT

How did the moon clock cope with this change – the first occurrence since we started?  I think the answer is 80 %.  Firstly, it did correctly recognise that BST had ended and this meant that npulse (the number of pulses needed at the end of each minute) was negative until the current time caught up with the displayed time.  So the s hand waited for one hour. (I did not get up to watch this).  In the morning, the moon was illuminated (it is near full moon with the moon up for 15 hours) and the s hand showed the right time. So far so good.

But the next test was whether the new rise and set times could be correctly displayed.  The problem is that on the 27/28th October, the rise time was 19:45 and the set time was 10:51, whereas on the 28/29th October, the rise time was 19:30 and the set time was 11:56. So the rise time was 15 minutes earlier on 28th than the previous day. This was caused by clocks going back an hour. Since the clock can’t go backwards, it would have to go forwards about 23 hours and 45 minutes, needing 43,000 pulses.  The algorithm decides that it is better to wait for 24 hours than to pulse this far, so it flashes the moon 500 times as an indicator.  Unfortunately, the counter in the flashmoon function  had somehow got deleted, so it was stuck in an infinite loop!  This seems like a silly error when the hard part was working, but it shows how easy it is to make a mistake when editing.  So I had to correct the error in the flashmoon function, adjust the rise and set times by hand and restart the clock.

Of course it will be a year until this situation arises again – when summer time comes into effect, times go forward and that’s not a problem.

Improvements to the Moon Clock Software

The moon clock seems to be keeping the correct time and to advance to the new rise and set times correctly.  So what improvements needed?

The main issue is that when running headless, there is no text output to tell you what the clock should be showing. This is a nuisance if you need to set the clock hands.  It is of course possible to direct the standard output to the console, but I have not been able to determine what buffer size is allocated, or how to flush it.  It could fill up after months or years. The same issue could apply to a file.  Moreover, directing to a file isn’t good because the output is buffered and only written when the buffer is full. Given the small amount of output from the clock, you’d never know what the current readings are.

It would be better to output to a GUI box nicely formatted to show just the expected information.  It is possible to do this in Python and my next step will be to try this out.

There is another small change I’d like to make.  We know that we are limited on the accuaracy that the h hand can show the set time, but there shouldn’t be much of an issue with the rise time.  Except that the rise time can be 15 hours or more after the previous set time. The present algorithm calculates the rise time display to be accurate at the time it is evaluated, 1.5 hours after the previous set time.  So the s hand can have moved by 13 or 14 hours before the rise time is reached, during which the m hand has moved about 13 or 14 minutes.  By allowing for this, we can get a more accurate rise time.  It can’t be completely accurate, of course, because the hands are all geared together.

 

The Moon Clock Software Published at last

The Moon Clock  has been soak-tested for a month now and seems to be running as designed.  So you can download it from the link below.

moon-clock-with-azimuth positioner18-08-18

Note that it has been given a file type of .txt (text).  If it is stored as a .py file, which is an executable file type, spam filters will  prevent it from downloading, as a precaution in case it contains malicious code.

As it is, you can review the code NotePad, or better in any code editor, such as NotePad++, which will colour the code to help you to follow its structure.

You will need to change the file type to .py in order for NotePad ++ or the IDLE interface in the Raspberry Pi, to be able to recognise it as Python code.  You can’t run it until you change the file type.

The code is fully commented and is fully described in my blog and on the Moon Clock Software page.  I hope it will be of interest and look forward to any (positive) feedback.

Final testing

I’ve been tidying up the moon clock operating software, removing redundant bits and clarifying the comments.  In the process, I’ve found some minor bugs, but who knows if I’ve introduced some new ones?  So I have decided to let it run for a while longer before releasing it.

Overall it seems to be running as intended.  The new moon is in two days time.  The clock is showing very little of the moon is visible: I wonder if it will last out two more days?  It will be a crucial test of the maths when the set time passes midnight. This will not be for another 11 days.  If this works correctly, we should be good to release the software fully.

I have previously said the clock motor was still dropping the occasional ‘tick’.  I decided to lengthen the ‘off’ time of the pulse from 17 to 20 ms. It looks as thought this might be enough to allow the rotor to settle properly.   I’ll need a 2-hour shift to give it a really good test of 3600 or more pulses, but over the last few days, the set time has only changed by a few minutes, so I’ll need to wait a bit longer.

Total Lunar Eclipse on Friday

Excitement in the Moon Clock World! There will be a total Lunar Eclipse on Friday 27 July 2018, visible from most of the UK, including London. 

Friday is the day of the Full Moon: lunar eclipses occur when the moon and sun are on the opposite sides of the earth, which is also the condition for Full Moon. However, the earth’s shadow must fall completely over the moon for a total eclipse. Because the moon’s orbit is slightly tilted relative to the earth’s orbit round the sun, it is only occasionally that the moon crosses the directly into the earth’s shadow at full moon. Usually, it is slightly above or below the shadow.  The eclipse will actually start before the moon has risen in London.  The eclipse will be total at 20:49 (moonrise) reaching a maximum at 21:21 and totality will end at 22:13.  During this period it will be very close to the horizon, so will only be visible from locations with a clear view to the South East.

The next total lunar eclipse visible in London  is on 21 January 2019.  This will occur during the small hours, starting at 2:36 am.  There will be a partial lunar eclipse in July 2019.

Lunar eclipses never occur alone:  there is always a solar eclipse at the new moon preceding or following the lunar eclipse, when the moon comes directly between the sun and the earth. Sometimes, there can be solar eclipses at both these new moons, and that is what is happening this summer!  Of course,  solar eclipses may not be total, and may not be visible across the whole globe.  Indeed, due to the relatively small amount of obscuration caused by the moon, total solar eclipses are much rarer than lunar eclipses: they do not last as long and are visible from a much smaller area – sometimes visible only over areas of ocean.

Apparently, there was a partial solar eclipse  on 13th July 2018 and there will be a partial solar eclipse on 11 August, but it will not be visible in London.   The next fairly full solar eclipse in London is not until the evening of 12th August 2026.

I gather that the ephem library is capable of calculating eclipses, but I shall not be adding this to the clock!  However, if I get a photo of the Lunar Eclipse, I will put it on  here.

Update
After more than a month of very hot, dry weather, guess what?  At 6 pm on the evening of the eclipse, there was a deluge!  No thunder, no lightning, just torrential rain for half and hour.  The sky had been clouding over since mid-afternoon, and I hoped that after this storm, it would clear.  It didn’t.  In hope, rather than in expectation, I headed for the North Downs above Reigate.  From here there was a clear view to the south, with the south downs, running along the channel coast, in the very far distance.  But in the far south, the sky was heavy.  And I could see flashes of lightning in those clouds.  An occasional spectacular fork that spanned tens of miles.  The distant storm moved off eastwards, so at least I stayed dry, but there was no chance of seeing the moon.  So I had to find this photo.

That’s been my luck with eclipses. I went to Cornwall for the August 1999 total solar eclipse, but I encountered the thickest cloud cover imaginable and it didn’t even get particularly dark at totality.  Is there something about the position of the sun and moon that causes the jet stream to deviate and drag wet weather in from the Atlantic?

The next lunar total eclipse is in January … not a good time for a clear sky, but we can live in hope!