Saturday, August 12, 2017

A Workaround for the MP3 Tagging Problem

I have found a workaround for the problem I was having. To quickly summarize, I use Logic Pro 9 to export projects to wave files. Then I want to convert them to fully tagged MP3 files.

I've been doing the conversion by hand using iTunes, and tagging in iTunes too. My projects tend to be in higher sample rates and bit depths -- for example, 24-bit, 96KHz. I would use Logic Pro to export the files to MP3 format, and tag them, but I have the following issues:

  1. In Logic Pro 9, if I ask Logic to bounce this project to an MP3 file, it does a long series of conversions and the end result is that I get an MP3 file at 48KHz, which is not what I want. There does not seem to be an option to force it to use 44.1KHz.
  2. In Logic Pro 9, you can also ask Logic to tag your file, but if you supply long fields, it will truncate them. I think this is because it supports a somewhat out-of-date version of the ID3 standard, while iTunes supports a later version which supports longer fields.

So why not just upgrade to Logic Pro X? Well, first, I'm not sure if this would actually fix the problem. At the moment, money is a bit tight. And I'm recording on a 2009 "3,1" Mac Mini. It works great, although it is slow. I don't think installing Logic Pro X on this machine is likely to make my projects easier. Most likely, Pro X eats considerably more memory and CPU and disc space than Logic Pro 9. So the plan is to make do, as much as possible, with what I have, until I can do some major upgrades, including replacing the computer.

Anyhow, to work around these issues with Logic Pro 9, I've been bouncing to a WAVE file, then bringing the 16-bit, 44.1KHz WAVE file into iTunes, creating an MP3 using iTunes, then tagging it by hand.

I wanted to see if I could do some of these steps on the command line, so I could do it in a script, a BBEdit worksheet, or even a Makefile. I'd like to automate that somewhat, not so much because I'm spending that much time producing podcasts, but because all these steps are error-prone. I'd also like to automate, at least partially, the generation of the entries in the podcast feed file. I'm always screwing up the time zone offset in dates, or forgetting to update the size of the file in bytes. It would be nice to have a script to do the grunt work, especially since I am often making versions to test, before I am happy enough with them to add them to the live podcast feed.

So I was trying to use LAME to do the encoding and tagging, but I discovered that iTunes would not import the "comment" field in the MP3 files created and tagged by LAME.

I asked Dan Benjamin on Twitter, and was happy that he replied, but he just wrote "Why not just bounce correctly from Logic?"

Maybe that works for him, but as I explained above, and in the link I sent him, it doesn't work for me, because I wind up with MP3 files encoded at 48KHz. I don't want to get too far into the weeds here, but I believe that making 48KHz MP3 files for podcasts are fairly pointless for most users, since they will need to be resampled on playback and resampling is lossy. For most listeners playing the files back on typical devices, 48KHz is a waste of storage space and will not provide a quality boost over a 44.1KHz file.

I also want to be able to use comment fields like this:

This work by Paul R. Potts is released under the Creative Commons Attribution-Noncommercial-Share Alike 3.0 Unported License (http://creativecommons.org/licenses/by-nc-sa/3.0/). See http://generalpurposepodcast.blogspot.com for more information.

And not have them truncated.

In the Hydrogen Audio forum I got a couple of useful replies, but no solutions as such. The root of the problem seems to be that iTunes silently fails to import certain types of comment fields in MP3 tags. I don't want to go too far down the ID3 rabbit hole, but it seems like if the language specified for the comment field is "XXX," iTunes will not import it.

When LAME writes an ID3v2 comment tag, it seems to set the language to either some unicode string, or if the command-line switch --id3v2-latin1 is used, to "XXX." There doesn't seem to be an option to set it to something else. In either case, iTunes will not import this field.

My wife Grace said "your encoder is lame."

I tried to use the id3v2 command-line tool to add the tags to my MP3 file instead. Since I'm making a script, I have no qualms about using two command-line tools instead of one, if that works. But id3v2 seems to have the same problem. It supports the --id3v2-only switch, but there does not seem to be any equivalent of the --id3v2-latin1 switch, so I can't get it to write an iTunes-compatible comment field either.

My workaround was to rebuild LAME, replacing several hard-coded instances of "XXX" with "eng." This is not exactly a bug to report, since I'm not sure that LAME is actually "wrong" per se, according to the standard. But I can't fix the issue in iTunes. And like it or not, if I want maximum compatibility, I have to generate files that work well with iTunes.

A better solution would probably be to give LAME more options. Specifically for this problem an option to set the comment language would be nice. I would hesitate to add a specific switch like --workarounditunes which just forced the comment language to "eng," but maybe the ability to set the comment language would be useful for other folks, while providing a workaround for this compatibility problem.

Ypsilanti, Michigan
August 12, 2017

Tuesday, August 08, 2017

Two Quick Technical Tidbits

Trump is blathering about about "fire, fury, and - frankly - power," and I'm trying to thinkg about something less distressing, so I'm going to write briefly about two technical issues I've come across, in the hopes that Google places this within reach of someone else looking for a solution.

Creating a Shared Memory Block with Keil µVision and the Atmel SAM4E16E

In my work, I have created firmware for the Atmel SAM4E16E, a nifty chip with an ARM core. My toolchain is Keil µVision version 5.23.

What I'm trying to do is conceptually very simple. We have a bootloader image and an application image. When the bootloader executes, I want it to put some specialized values into a block of memory, at a fixed, known address. When the bootloader executes to run the main application, I want the main application to read some information from that block of memory.

The challenge is not so much in writing the C code to handle this -- that's quite easy. The challenge is configuring your tools to get out of your way. In particular, you want the linker to set aside the memory in the right place, and the startup code to leave it alone (not zero it out). Documentation on this is a bit sparse and confusing. It took me quite a bit of trial and error to get it working. Along the way I discovered that the tools are both more poorly-documented and less robust than I hoped. But it did work, and here's how.

First, I created a common header file for describing the memory structure. It looks something like this:

typedef struct Bootloader_Shared_Memory_s
{
    uint32_t prefix;
    uint32_t version;
    uint32_t unused0;
    uint32_t unused1;
    uint32_t unused2;
    uint32_t unused3;
    uint32_t unused4;
    uint32_t unused5;
    uint32_t unused6;
    uint32_t unused7;
    uint32_t suffix;

} Bootloader_Shared_Memory_t;

extern Bootloader_Shared_Memory_t bootloader_shared_memory;

That just gives us a series of 32-bit words in memory. I want to set a prefix and suffix value to some special values that I will look for, to see if the shared memory block looks like it was configured as I expect. It is very unlikely that the prefix and suffix would have these values unless they were deliberately put there.

#define BOOTLOADER_SHARED_DATA_PREFIX ( 0x00ABACAB )
#define BOOTLOADER_SHARED_DATA_SUFFIX ( 0xDEFEEDED )

Then I just define a function that will configure the shared memory, instead of specifying inital values in the definition. That's because I don't want the compiler to treat this block as part of its initialized memory (the ".data" section of memory). See: https://en.wikipedia.org/wiki/Data_segment

void Configure_Bootloader_Shared_Memory( void );

The bootloader code calls this function, which looks like this:

void Configure_Bootloader_Shared_Memory( void )
{
    bootloader_shared_memory.prefix = BOOTLOADER_SHARED_DATA_PREFIX;
    bootloader_shared_memory.version = ( ( BOOTLOADER_REVISION_HIGH   << 16 ) |
                                         ( BOOTLOADER_REVISION_MIDDLE << 8  ) |
                                         ( BOOTLOADER_REVISION_LOW )            );
    bootloader_shared_memory.suffix = BOOTLOADER_SHARED_DATA_SUFFIX;

}

Where BOOTLOADER_REVISION_HIGH etc. are #defines which specify the current version number.

Now, we've declared the data structure and made a function that operates on it. We need to define the data structure. I do this in a separate C file:

/*
    Highest possible base address (last RAM address is 0x2001FFFF), masked to zero off two low bits of address for 4-byte alignment
*/
#define BOOTLOADER_SHARED_MEMORY_BASE_ADDRESS ( ( 0x20020000 - sizeof( Bootloader_Shared_Memory_t ) ) & 0xFFFFFFFC )

/*
    Note: comes out to 0x2001FFD4, the data structure is 0x2C bytes long; will have to change the scatter file if our data structure changes
*/
__attribute__((at(BOOTLOADER_SHARED_MEMORY_BASE_ADDRESS),zero_init))

Bootloader_Shared_Memory_t bootloader_shared_memory;

Note that non-standard attribute. It creates a separate memory area to be passed to the linker. It specifies the base address. The *zero_init* is confusing; suffice it to say that I found it mentioned in ARM's documentation as a workaround. It doesn't mean, apparently, "initialize this block of memory to zero." It seems to mean "do no initialization of this block of memory."

Now, you might think that we've told the linker enough, but apparently we haven't. We have to use a scatter-gather file. This is a file with the extension .sct. Now, we're getting pretty deep in the weeds. The documentation on this file format is pretty sparse. I found few useful examples. The standard way to build my project seemed to result in the toolchain creating its own scatter-gather file, which looks like this:

LR_IROM1 0x00400000 0x000E0000  {    ; load region size_region
  ER_IROM1 0x00400000 0x000E0000  {  ; load address = execution address
   *.o (RESET, +First)
   *(InRoot$$Sections)
   .ANY (+RO)
  }
  RW_IRAM1 0x20000000 0x00020000  {  ; RW data
  .ANY (+RW +ZI)
  }
}

Apparently this specifies the memory regions which the linker will use. It hunts through them applying some rules to determine if each code or data object can go in a region. The RW_IRAM1 is our RAM.

We can create our own scatter-gather file by starting with this tool-generated file and adding a region:

LR_IROM1 0x00400000 0x000E0000  {    ; load region size_region
  ER_IROM1 0x00400000 0x000E0000  {  ; load address = execution address
   *.o (RESET, +First)
   *(InRoot$$Sections)
   .ANY (+RO)
  }
  bootloader_shared 0x2001FFD4 UNINIT 0x0000002C {
    Bootloader_Shared_Memory.o (+RW +ZI)
  }
  RW_IRAM1 0x20000000 0x00020000  {  ; RW data
   .ANY (+RW +ZI)
  }
}

I've added a special region which will fit any data object in the BootloaderSharedMemory.o object file. I've added the special obscure "UNINIT," which I also found mentioned in some somewhat obscure ARM documentation.

Anyway, to make a long story short, this seems to work for me, while any variation of it does not work for me. If you mess up your scatter-gather file a bit, you can see strange behavior -- for example, my image file became over a gigabyte in size at one point. The toolchain doesn't seem very robust as far as error-checking goes, here. If you don't use both the special "UNINIT" and "zero_init" specifiers, things don't work right. It was quite confusing.

To make the build process use the custom scatter-gather file, I had to turn off the checkbox in the project's linker options that reads "Use Memory Layout from Target Dialog." Then I had to provide a path to the scatter file. The linker options dialog should show a -scatter option with a path to your custom scatter-gather file. It isn't enough just to provide a file.

You should be able to tell if it worked right by looking at the link map. It shows the address of the shared memory object. The attribute created the special memory region named with its address.

.ARM.__AT_0x2001FFD4                     0x2001ffd4   Section       44  bootloader_shared_memory.o(.ARM.__AT_0x2001FFD4)

Then, in my application firmware, which reads the information from the shared memory object, I have to configure the linker to use the same scatter-gather file. Then, elsewhere in my code, I can look for the special prefix and suffix to decide if I want to treat the version field in the structure as legitimate data:

if ( ( BOOTLOADER_SHARED_DATA_PREFIX == bootloader_shared_memory.prefix ) && 
     ( BOOTLOADER_SHARED_DATA_SUFFIX == bootloader_shared_memory.suffix )    )
{
    /*
        Do something with bootloader_shared_memory.version
    */
}

This may not be the absolute best or simplest way to achieve what I set out to achieve -- if I've done something glaringly wrong, please leave a comment. But it seems to work, and so I hope someone else might find this useful.

Tagging MP3 files with LAME (Unsolved Mysteries)

I've been trying to simplify my podcast workflow. My old workflow, in part, looked like this:

1. Bounce a track from Logic Pro at a 24 bit/96KHz. Tell Logic to do the conversion to 16 bit/44.1KHz.
2. Don't ask Logic to make an MP3 because if I do it this way, it creates a 48KHz MP3, and I don't want that.
3. Import the 16 bit/44.1KHz track into iTunes.
4. Have iTunes create the MP3 file (annoying because I have to configure the conversion using CD import settings).
5. Use the iTunes "Get Info" tag editor to tag the MP3 file.
6. Find where iTunes put it using "Show File in Finder" and get it back out, then that becomes the tagged file I upload to my server.

I'd rather do it like this instead:

1. Bounce a track from Logic Pro at a 24 bit/96KHz. Tell Logic to do the conversion to 16/44.1.
2. Use a little shell script to do both the conversion to MP3 conversion and tagging with LAME.

This seems to work fine, in that it creates the MP3 file with the bit rate I expect, and the tags I expect, as seen in the MacOS X Finder. That is, if I do "Get Info" on the MP3 file, it shows all the tag I added, including the comments field.

However, if I then bring that MP3 file into iTunes, either just by importing it or by downloading it via my podcast feed, the tags all work except for the comment field. The comment field appears to be empty.

If I look at the LAME-tagged file with Windows file explorer using "Properties," the comment tag shows up as expected. When I look at the iTunes-tagged file, it shows the "Comments" field as either containing 0, or some hexadecimal garbage. In other words, with the Windows file explorer the iTunes-tagged file appears broken, while the LAME-tagged file looks fine. So clearly there is some kind of compatibility problem going on.

Using a Windows program called "Tag Clinic," I can see a difference -- it looks like the LAME-created file has a comment tag that is ID3v2.3 with the Language field showing nothing. The iTunes-tagged file shows as ID3v2.2 with the Language field showing English.

So the question seems to be: can I make LAME export a tag that iTunes will read? If not, is this enough of a deal-breaker that I have to stop using LAME? I would like to create MP3 files with maximum compatibility, and a lot of folks (including me) use iTunes. I care less about "who is broken/buggy/not honoring the standard" as "how can I make my workflow easier while still creating podcast files that will work well for most users."

I didn't get any suggestions yet on Reddit. I had a comment on the Hydrogen Audio forum, suggesting that I "try forcing 2.2" (use ID3 version 2.2 comment tags). However, I don't see a way to force LAME to do that. There's a --id3v2-only switch, but I think that turns off the use of version 1 tags.

I'm left wondering if there is a different command-line utility that will encode and tag my file (ffmpeg, maybe?) Or if I should skip having LAME add the comment tag and instead use a separate command-line utility, id3tag.

But I can't help feeling that I'm missing something -- can the most popular command-line encoding tool really not create MP3 files that play nicely with iTunes? It seems like there must be a way to make this easier. If you've got a suggestion, please leave a comment. Comments are moderated, so I'll have to approve them.

Ypsilanti, Michigan
August 8, 2017

Thursday, October 06, 2016

Fixed Point Math with AVR-GCC

Wow, I see that it has been a long time since my last post. Sorry about that. I've been very busy. I have lots to talk about. I'd like to write about reading encoders, and I'd like to write about communicating with EEPROM chips that use two-wire protocols (I2C-like) as opposed to SPI-like protocols. But in the meantime I hope this short post will be useful to someone.

Embedded C

I recently had reason to do some non-integer math on a small microcontroller, a member of the Atmel ATtiny series. Floating point math on this chip is pretty much out of the question; there is no floating-point hardware. I think some of the chips in this family are big enough to hold floating-point library functions, but they will certainly eat up an enormous amount of the available program space, and given that they are eight-bit microcontrollers in most ways -- the registers are 8 bits wide -- it is probably best to just avoid floating point.

So I began looking into fixed-point math. It is always possible to roll your own code for this kind of thing, but I thought I would see if I could take advantage of existing, debugged library code first. I found some free software libraries online, but because I develop code that runs in commercial products, I was not really happy with their license terms. It also was not very clear how to use them or whether they would fit on the ATtiny chips.

I discovered that there is, in fact, a standard for fixed-point types in C. It has not been widely adopted, and like the C standard itself it is a little loose in parts, in that it doesn't dictate the numeric limits of types, but rather specifies a range of acceptable sizes. And it turns out that my toolchain supports this standard, at least in part.

I won't try to describe everything covered in the Embedded C document. I'll spare you my struggle trying to find adequate documentation for it or determine how to do certain things in an implementation that doesn't implement everything in the Embedded C document.

Instead I will try to do something more modest, and just explain how I managed to use a couple of fixed-point types to solve my specific problems.

You can find more information on the Embedded C standard here: https://en.wikipedia.org/wiki/Embedded_C

The actual Embedded C standards document in PDF form can be found here (note: this is a link to a PDF file): http://www.open-std.org/JTC1/SC22/WG14/www/docs/n1169.pdf.

At the time of this writing, this seems to be the latest version available, dated April 4, 2006. The document indicates a copyright, but unlike the C and C++ standards, it looks like you can download it for no cost, at least at present.

avr-gcc

The compiler I'm using is avr-gcc. My actual toolchain for this project is Atmel Studio version 7.0.1006. Atmel Studio is available for download at no cost. The avr-gcc toolchain that Atmel Studio uses under the hood is available in other toolchains and as source code. I'm not going to try to document all the ways you can get it, but you can find out more here: https://gcc.gnu.org/wiki/avr-gcc.

As I understand it, these Embedded C extensions are not generally in across other versions of GCC.

The Basics of Fixed Point Types in Embedded C

I'm assuming I don't have to go into too much detail about what fixed-point math is. To put it briefly, fixed point types are like signed or unsigned integral types except there is an implicit binary point (not a decimal point, a binary point). To the left of that binary point, the bits indicate ascending powers of two as usual: 1, 2, 4, 8, etc. To the right of that binary point, the bits indicate fractional powers of two: 1/2, 1/4, 1/8.

The Embedded C extensions for fixed-point math came about, I believe, at least originally because many microcontrollers and digital signal processors have hardware support for fixed-point math. I've used DSPs from Motorola and Texas Instruments that offered accumulators for fixed-point math in special wide sizes, such as 56 bits, and also offered saturation arithmetic. Using these registers from C required special vendor-specific compiler support. If they were supported instead using Embedded C types, programmers would have a better shot at writing portable code.

There are a couple of basic approaches to these types mentioned in the standard. There are fractional types, indicated with the keyword Fract, with values between -1.0 and 1.0, and types that have an integral part and a fractional part, indicated with the keyword Acum. It is expected that implementations will give these aliases, like fract and accum, but I think the authors did not want to introduce potential name clashes with existing code.

The standard specifies the minimal formats for a number of types. For example, unsigned long accum is provides a minimum of 4 integer bits and 23 fractional bits. In our implementation, unsigned long accum actually provides 32 integral bits and 32 fractional bits. It maps to an underlying type that can hold the same number of bits. On this platform, that underlying type is unsigned long long, which on this platform is 64 bits.

Accumulator Types

For my algorithms, I don't have much interest in the Fract types and I'm going to use only the Accum types. I would have more interest in Fract_ types if there were standard ways available to multiply them together. In that case I could use a Fract_ type as a scale factor to apply to a large-ish integer in Accum representation. For example, let's say I want to generate an unsigned binary value to send to a DAC that accepts 18-bit values. I could create a value of an Accum type that represents the largest 18-bit value, and scale this by a _Fract value indicating a fraction to apply.

The advantage of this approach would be, I thought, that I would use types that were only as wide as I needed, resulting in less code. However, since this does not seem easy or convenient to do, in my own code I am using only _Accum types at present.

And, in fact, I'm using only unsigned _Accum types, specifically types that have a 16-bit unsigned integer value and a 16-bit fractional value (aka “16.16”), unsigned accum, and a 32-bit unsigned integer value and a 32-bit fractional value (aka “32.32”), unsigned long accum. The underlying types used to implement unsigned accum and unsigned long accum are unsigned long (32 bits) and unsigned long long (64 bits).

Fixed Point Constants

There are new suffixes to allow specifying fixed-point constants. For example, instead of specifying 15UL (for unsigned long), one can write 15UK for an unsigned accum type, or 15ULK for an unsigned long accum type. One can specify the fractional part, like 1.5UK.

On this platform, 1.5UK assigned to a variable of unsigned accum type will produce the 16.16 bit pattern 0000 0000 0000 0001 1000 0000 0000 0000 (hex 00018000), where the most significant 16 bits represent the integer part, and the least significant 16 bits represent the fractional part.

Accuracy

For our purposes we will mostly be using the integer results of fixed-point calculations. We don’t need to use the FX_FULL_PRECISION pragma; error of 2 ULPs for multiplication and division operations is fine.

A Very Simple Example

Here's a small program that shows a very simple calculation using unsigned accum types. I created a simple project in Atmel Studio that targets the ATtiny 841 microcontroller, which has 512 bytes of SRAM and 8 KiB of flash memory for programs. Today I'm not using a hardware debugger or attached chip. It is possible to configure the project's "Tool" settings to use a simulator instead of a hardware debugger or programmer.

#include <avr/io.h>
#include <stdfix.h>

static unsigned accum sixteen_sixteen_half = 0.5UK;
static unsigned accum sixteen_sixteen_quarter = 0.25UK;
static unsigned accum sixteen_sixteen_scaled;

int main(void)
{
    sixteen_sixteen_scaled = sixteen_sixteen_half * sixteen_sixteen_quarter;
}

We can watch this run in the debugger. In fact, this is the reason for including the volatile keyword in the variable declarations. Even with optimizations turned off, the compiler will still aggressively put variables in registers and avoid using memory at all if it can. While I don't seem to be able to use watches on these variables, as I can when using a hardware debugger and microcontroller, I can see the values change in memory as I step through the program. The values are organized as little-endian. Translating this, I can see that sixteen_sixteen_half shows up as 0x00008000, sixteen_sixteen_quarter shows up as 0x00004000, and the result of the multiplication operation, sixteen_sixteen_scaled, is assigned 0x00002000, representing one-eighth.

Code Size

If I bring up the Solution Explorer window (via the View menu in Atmel Studio), I can take a look at the output file properties by right-clicking. The generated .hex file indicates that it is using 310 bytes of flash. If I do the same calculation using float types, the library support for floating-point multiplication makes the flash use 580 bytes.

What happens if I scale up to a larger type? Well, if I change my unsigned accum declarations to use unsigned long accum, suddenly my flash usage goes up to 2776 bytes. That's a lot given that I have 8192 bytes of flash, but it still leaves me quite a bit of room for my own program code.

A Few Techniques

Let's say we want to scale a value to send to a linear DAC. Our DAC accepts 18-bit values. That means we can send it a value between 0x0 and 0x3FFFF.

To work directly with an _Accum type that will represent these values, we have to use an unsigned long accum. To declare an unsigned long accum variable that is initialized from an unsigned long variable, I can just cast it:

unsigned long accum encoder_accum = ( unsigned long accum )encoder_val;

We can also cast from a shorter integral type -- for example, from an unsigned accum -- and get the correct results. Beware of mixing signed and unsigned types! (As you always should, when working in C).

We can do math on our unsigned long accum types using the usual C math operators.

Let's say we want to get the unsigned long accum value converted back to an integral type. How would we do that? We use bitsulk to get the bitwise value (this is actually just a cast operation under the hood). Because we're going to truncate the fractional part, I add 0.5ULK first.

unsigned long val = bitsulk( encoder_accum + 0.5ULK ) >> ULACCUM_FBIT;

If we want the remainder as an unsigned long accum, we can get it. Remember that the fractional part of the accumulator type be [0.0..1.0] (that is, inclusive of zero, exclusive of 1). Note that the use of the mask here is not very portable, although there are some tricks I could do to make it more portable, but for now, I am more concerned about readability.

unsigned long accum remainder = ulkbits( bitsulk( encoder_accum ) & 0xFFFFFFFF );

The ulkbits and bitsulk operations are just casts, under the hood, so this boils down to a shift and mask.

The Embedded C specification defines a number of library functions that work with the fractional and accumulator types. For example, abslk() will give absolute value of an unsigned long accum argument. There are also rounding functions, like roundulk(). I have not actually had need of these. They seem to be supported in avr-gcc, but so far I have not needed them.

Conclusion

I hope this very brief tutorial may have saved you some time and aggravation in trying to use these rather obscure, but very useful, language features. If you come across anything interesting having to do with avr-gcc's support for the Embedded C fixed-point types, please leave a comment!

Saginaw, Michigan
October 6, 2016

Friday, February 26, 2016

SPI Communications with the Arduino Uno and M93C46 EEPROM: Easy, Fun, Relaxing

When I write code for an embedded microprocessor, I frequently need to use communications protocols that allow the micro to communicate with other chips. Often there are peripherals built in to the micro that will handle the bulk of the work for me, freeing up micro clock cycles and allowing me to write fewer lines of code. Indeed, the bulk of modern microcontroller datasheets is usually devoted to explaining these peripherals. So, if you aren't trying to do anything unusual, your micro may have a peripheral that will do most of the work for you. There might be a pre-existing driver library you can use to drive the peripheral. But, sometimes, you don't have a peripheral, or it won't do just what you need it to do, for one reason or another In that case, or if you just want to learn how the protocols work, you can probably seize control of the GPIO pins and implement the protocol yourself.

That's what I will do, in the example below. I will show you how to implement the SPI (Serial Peripheral Interface) protocol, for communicating with an EEPROM. I've used SPI communication in a number of projects on a number of microcontrollers now. The basics are the same, but there are always issues to resolve. The SPI standard is entertaining keeps you on your toes, precisely because it is so non-standard; just about every vendor extends or varies the standard a bit.

The basics of SPI are pretty simple. There are four signals: chip select, clock, incoming data, and outgoing data. The protocol is asymmetrical; the microcontroller is usually the master, and other chips on the board are slaves -- although it would be possible for the micro to act as a slave, too. The asymmetry is because the master drives the chip select and clock. In a basic SPI setup, the slaves don't drive these signals; the slave only drives one data line. I'll be showing you how to implement the master's side of the conversation.

Chip select, sometimes known as slave select from the perspective of the slave chip, is a signal from the master to the slave chip. This signal cues the slave chip, informing the chip that it is now "on stage," ready for its close-up, and it should get ready to communicate. Whether the chip select is active high, or active low, varies. Chip select can sometimes be used for some extra signalling, but in the basic use case the micro set the chip select to the logically active state, then after a short delay, starts the clock, runs the clock for a while as it sets and reads the data signals, stops the clock, waits a bit, and turns off the chip select.

Here's a picture showing the relationship between clock and chip select, as generated by my code. Note that I have offset the two signals slightly in the vertical direction, so that it is easier to see them:

The clock signal is usually simple. The only common question is whether the clock is high when idle, or low when idle. Clock speeds can vary widely. Speeds of 2 to 10 MHz are common. Often you can clock a part much slower, though. CMOS parts can be clocked at an arbitrarily slow speed; you can even stop the clock in the middle of a transfer, and it will wait patiently.

What is less simple is the number of clocks used in a transaction. That can become very complex. Some parts use consistent transfer lengths, where for each transaction, they expect the same number of clock cycles. Other parts might use different numbers of clock cycles for different types of commands.

From the perspective of the slave, the incoming data arrives on a pin that is often known, from the perspective of the microcontroller, as MOSI (master out, slave in). This is again a simple digital signal, but the exact way it is interpreted can vary. Essentially, one of the possible clock transitions tells the slave to read the data. For example, if the clock normally idles low, a rising clock edge might signal the slave to read the data. For reliability, it is very important that the master and slave are in agreement about which edge triggers the read. Above all, you want to avoid the case where the slave tries to read the incoming data line on the wrong edge, the edge when the master is allowed to change it. If that happens, communication might seem to work, but it works only accidentally, because the slave just happens to catch the data line slightly after it has changed, and it may fail when the hardware parameters change slightly, such as when running at a higher temperature.

Let me be as clear as I can: when implementing communication using SPI, be certain you are very clear about the idle state of the clock line, and which clock transition will trigger the slave to read the data line. Then, make sure you only change the data line on the opposite transition.

Terminology surrounding SPI transactions can be very confusing. According to Wikipedia and Byte Paradigm, polarity zero means the clock is zero (low) when inactive; polarity one means the clock is one (high) when inactive.

Phase zero means the slave reads the data on the leading edge, and the master can change the value on the trailing edge, while phase one means the slave reads the data line on the rising edge, and the master changes the data line on the falling edge).

But some Atmel documentation (like this application note PDF file) uses the opposite meaning for "phase," where phase one means the slave reads data on the leading edge.

Because of this confusion, in my view it is best not to specify a SPI implementation by specifying "polarity" and "phase." So what would be clearer?

Aardvark tools use the terms "rising/falling" or "falling/rising" to describe the clock behavior, and "sample/setup" or "setup/sample" to indicate the sampling behaviors. I find this to be less ambiguous. If the clock is "rising/falling," it means that the clock is low when idle, and rises and then falls for each pulse. If the "sample" comes first, it means that the slave should read the data line on the leading edge, and if the "setup" comes first, it means that the slave should read the data on the trailing edge.

Here's a picture of my clock signal along with my MOSI (master out, slave in) signal. This SPI communication variant is "rising/falling" and "sample/setup." In order to allow the slave to read a valid bit on the leading clock edge, my code sets the MOSI line to its initial state before the rising edge of the first clock pulse. Again, I have offset the signals slightly in the vertical direction, so that it is easier to see them:

In the screen shot above, the master is sending nine bits: 100110000. Each bit is sampled on the rising clock edge. On the first rising clock edge, the MOSI line (in blue) is high. On the second rising clock edge, the MOSI line is low.

From the perspective of the slave, the outgoing data is sent on a pin that is often known as MISO (master in, slave out). This works in a similar way as the incoming data, except that the slave asserts the line.

When the master sends data to the slave, the master turns on the chip select (whether that means setting it low, or setting it high), changes the MOSI line and clock as needed, and then turns off the chip select.

When the master receives data from the slave, the behavior is slightly more confusing. To get data from the slave, the master has to generate clock cycles. This means that it is also sending something, depending on how it has set the MOSI line. During the read operation, what it is sending may consist of "I don't care" bits that the slave will not read. Receiving data can sometimes require one transaction to prepare the slave for the read operation, and then another to "clock in" the data. Sometimes a receive operation may be done as one transaction, but with two parts: the master sends a few bits indicating a read command, and then continues to send clock cycles while reading the slave's data line. Sometimes there are dummy bits or extra clock cycles in between the parts of this transaction.

Here's a picture that shows a read operation. I'm showing clock and MISO (mmmm... miso!) This shows a long transaction where the master sends a request (the MOSI line is not shown in this picture) and then continues to generate clock pulses while the slave toggles the MISO line to provide the requested data.

Now let's look at my hardware and software. I wrote some code to allow an Arduino Uno to communicate with a serial EEPROM chip. The chip in question is a M93C46 part. This is a 1Kb (one kilobit, or 1024 bits) chip. The parts are widely available from different vendors. I have a few different through-hole versions that I got from various eBay sellers; in testing them, they all worked fine. The datasheet I used for reference is from the ST Microelectronics version of the part.

These parts all seem to have similar pinouts. Pin 1 is the chip select, called slave select in the STM documentation. Pin 2 is the clock. Pins 3 and 4 are data pins. On the other side of the chip, there is a pin for +5V or +3.3V, a pin for ground, an unused pin presumably used by the manufacturer for testing, and a pin identified as ORG (organization), which determines whether the data on the chip is organized into 64 16-bit words, or 128 8-bit bytes.

There are other versions of this chip; the 1Kb is only one version. The command set differs slightly between sizes, but it should be pretty easy to adapt my example to a different-sized part. A full driver would be configurable to handle different memory sizes. It would not be hard to implement that, but for this example I am keeping things simple.

Here's my simple circuit, on a prototype shield mounted to an Arduino Uno:

Here's a simple schematic showing the Arduino pins connected to the EEPROM chip:

I'm not much of an electrical engineer, but that should convey that pin 1, usually marked with a little dot or tab on the chip, is on the lower right. We count pins counter-clockwise around the chip. So pin 5 goes to ground (I used the ground next to the data pins; that is the green wire going across the board). Make sure you are careful to connect the right pins to power and ground, or you can let the magic smoke one of these little EEPROM chips, and maybe disable your Arduino board, too, perhaps permanently (you'll never guess how I know this!)

I also have three LEDs connected to three more pins, connected through 220 ohm resistors, with the negative side of the LEDs going to a ground pin on the left side of the prototype board. Those are not required; they are there solely to create a simple busy/pass/fail display. You can use the serial monitor, if the Arduino is attached to your computer, or whatever other debugging method is your favorite.

I have done this kind of debugging with elaborate, expensive scopes that have many inputs and will decode SPI at full speed. That is very nice, but you don't necessarily need all for a simple project like this. I got this project working using a Rigol two-channel scope. I was not able to capture a trace of all our lines at once using this scope, but I didn't need to. With two channels, I could confirm that the chip select and clock were changing correctly with respect to each other. Then I could look at the MOSI along with the clock and verify that the data was changing on the expected clock transition. Then I could look at the MISO along with the clock to verify the bits the Arduino was getting back from the serial EEPROM chip. Here's my modest setup, using a separate breadboard rather than a shield:

Here's a view of a SPI conversation with the EEPROM chip: a write operation, followed by a read operation to verify that I can get back what I just wrote. This shows clock and MOSI, so we don't see the slave's response, but you can see that the second burst has a number of clock cycles where the master is not changing the data line. Those are "don't care" cycles where the master is listening to what the slave is saying. Note also that I am running this conversation at a very slow clock speed; each transition is 1 millisecond apart, which means that my clock is running at 500 Hertz (not MHz or even KHz). I could certainly run it faster, but this makes it easy to see what is happening, if I toggle an LED along with the chip select to show me when the master is busy.

Now, here's some code.

You don't have to use these pins, but these are the ones I used.

#define SLAVESELECT 10 /* SS   */
#define SPICLOCK    11 /* SCK  */
#define DATAOUT     12 /* MOSI */
#define DATAIN      13 /* MISO */

Here's a "template" 32-bit word that holds a 16-bit write command.

#define CMD_16_WRITE ( 5UL  << 22 )
#define CMD_16_WRITE_NUM_BITS ( 25 )

This defines a 25-bit command. There is a start bit, a 2-bit opcode, a six-bit address (for selecting addresses 0 through 63), and 16 data bits.

To use this template to assemble a write command, there's a little helper function:

uint32_t assemble_CMD_16_WRITE( uint8_t addr, uint16_t val )
{
    return ( uint32_t )CMD_16_WRITE   |
           ( ( uint32_t )addr << 16 ) |
             ( uint32_t )val;
}

Now we need a function that will send that command. First, let's start with a function that will send out a sequence of bits, without worrying about the chip select and final state of the clock.

void write_bit_series( uint32_t bits, uint8_t num_bits_to_send )
{
    uint8_t num_bits_sent;

    for ( num_bits_sent = 0; num_bits_sent < num_bits_to_send;
          num_bits_sent += 1 )
    {
        digitalWrite( SPICLOCK, LOW );
        digitalWrite( DATAOUT, bits & ( 1UL <<
            ( num_bits_to_send - num_bits_sent - 1 ) ) ? HIGH : LOW );
        delay( INTER_CLOCK_TRANSITION_DELAY_MSEC );
        digitalWrite( SPICLOCK, HIGH );
        delay( INTER_CLOCK_TRANSITION_DELAY_MSEC );

    }

}

This maps the bits to the DATAOUT (or MISO) line. We change the data line on the falling edge of the clock. We aren't using a peripheral to handle the SPI data; we just "bit bang" the outputs using a fixed delay.

Here's a function that will send a command that is passed to it. It works for write commands:

void write_cmd( uint32_t bits, uint8_t num_bits_to_send )
{
    digitalWrite( SLAVESELECT, HIGH );
    delay ( SLAVE_SEL_DELAY_PRE_CLOCK_MSEC );

    write_bit_series( bits, num_bits_to_send );

    /*
        Leave the data and clock lines low after the last bit sent
     */
    digitalWrite( DATAOUT, LOW );
    digitalWrite( SPICLOCK, LOW );

    delay ( SLAVE_SEL_DELAY_POST_CLOCK_MSEC );
    digitalWrite( SLAVESELECT, LOW );

}

That's really all you need to send out a command. For example, you could send a write command like this:

write_cmd( assemble_CMD_16_WRITE( addr, write_val ), CMD_16_WRITE_NUM_BITS );

Note that before you can write successfully, you have to set the write enable. My code shows how to do that. Basically, you just define another command:

#define CMD_16_WEN   ( 19UL <<  4 )
#define CMD_16_WEN_NUM_BITS   (  9 )

write_cmd( ( uint16_t )CMD_16_WEN, CMD_16_WEN_NUM_BITS );

This EEPROM chip will erase each byte or word as part of a write operation, so you don't need to perform a separate erase. That may not be true of all EEPROM chips.

To read the data back, we need a slightly more complex procedure. Our read command uses the write_bit_series function to send out the first part of the read command, then starts clocking out "don't care" bits and reading the value of the MOSI line:

uint16_t read_16( uint8_t addr )
{
    uint8_t num_bits_to_read = 16;
    uint16_t in_bits = 0;
    uint32_t out_bits = assemble_CMD_16_READ( addr );

    digitalWrite( SLAVESELECT, HIGH );
    delay ( SLAVE_SEL_DELAY_PRE_CLOCK_MSEC );

    /*
        Write out the read command and address
    */
    write_bit_series( out_bits, CMD_16_READ_NUM_BITS );

    /*
        Insert an extra clock to handle the incoming dummy zero bit
    */
    digitalWrite( DATAOUT, LOW );

    digitalWrite( SPICLOCK, LOW );
    delay( 1 );

    digitalWrite( SPICLOCK, HIGH );
    delay( 1 );

    /*
        Now read 16 bits by clocking. Leave the outgoing data line low.
        The incoming data line should change on the rising edge of the
        clock, so read it on the falling edge.
    */
    for ( ; num_bits_to_read > 0; num_bits_to_read -= 1 )
    {
        digitalWrite( SPICLOCK, LOW );
        uint16_t in_bit = ( ( HIGH == digitalRead( DATAIN ) ) ? 1UL : 0UL );
        in_bits |= ( in_bit << ( num_bits_to_read - 1 ) );
        delay( INTER_CLOCK_TRANSITION_DELAY_MSEC );

        digitalWrite( SPICLOCK, HIGH );
        delay( INTER_CLOCK_TRANSITION_DELAY_MSEC );
    }

    /*
        Leave the data and clock lines low after the last bit sent
     */
    digitalWrite( DATAOUT, LOW );
    digitalWrite( SPICLOCK, LOW );

    delay ( SLAVE_SEL_DELAY_POST_CLOCK_MSEC );
    digitalWrite( SLAVESELECT, LOW );

    return in_bits;

}

And that's the basics. To test this, I put an EEPROM chip on a breadboard and just wired up the pins as specified in the code. Check your datasheet to determine if you can power the part with 5V or 3V. The chips I got seem to work fine with either, although if you are testing with a scope, you might want to use 5V so that the data out you get back from the chip has the same level as the 5V Arduino outputs.

You can find the full sketch on GitHub here.

Good luck, and if you found this useful, let me know by posting a comment. Comments are moderated, so they will not show up immediately, but I will post all (non-abusive, non-spam) comments. Thanks for reading!

Friday, January 01, 2016

Star Wars: The Force Awakens

This review contains many spoilers.

I want to start out by saying that I really was expecting, even hoping, to dislike The Force Awakens.

Entering the theater a cynical, somewhat bitter middle-aged man, I fully expected to be able to take my distaste for the other work of J. J. Abrams (particularly, his atrocious 2009 Star Trek reboot), and Disney, and recycled nostalgia in general, and throw it directly at the screen.

An original fan of Star Wars -- I saw the first one perhaps a dozen times in the theater -- and pretty much agree with the critical consensus about the prequels. Their utter failure led me to believe that the things I loved most about Episode IV had, for the most part, little to do with big-budget filmmaking, but were the result of giving a bunch of really brilliant costume and set designers and cinematographers and editors and sound designers a lot of creative control and a relatively low budget -- a situation unlikely to be replicated in a truly big film, an important investment where none of the investing parties would want to take any significant risks.

I was wrong, and I'm still somewhat troubled by that. Is The Force Awakens a good movie, or was I just primed by my age and the bad experience with the prequels to suck up something relatively bad and call it good, simply because it lacks the awfulness of the prequels, and smells a lot like the 1977 original? I don't think I can actually answer that question, definitively, at least not easily because really taking that up requires me to think critically about the original 1977 Star Wars, something I find hard to do, given the way the film imprinted itself upon my nine-year-old brain. Is it really all that and a bag of chips? Or did it just land at the right time to be the formative movie of my childhood?

One of my sons is nine, by the way. He enjoyed the new movie, but I don't think it blew his mind the way the original Star Wars blew mine, simply because we have, ever since 1977, lived in the era that had Star Wars in it.

To be clear -- it's not the case that there weren't big action movies back then, and big science fiction movies back then. We had movies like 2001: a Space Odyssey, which also formed my tastes. We had Silent Running. We had Logan's Run. But it would be impossible to overstate the shock wave of Star Wars -- the innovative effects, editing, and yes, even marketing. We just can't go back to that world. He's seen a lot of things that are Star Wars-ish, while in 1977, I never had.

And make no mistake, the new Star Wars is, most definitely, Star Wars-ish, in the way that the prequels were not. The world of the prequels was too clean, to sterile, too political, and too comic. Star Wars may have been the single most successful blending of genres ever attempted; a recent article called it "postmodern," and I think that is correct. The prequels might have been attempts at post-modern, too, but they seem to have a different set of influences, and just seem, in every respect, to have been assembled lazily, and without artfulness. For just one example, see how one of the prequel lightsaber battle scenes was actually filmed.

The Force Awakens follows the 1977 formula so closely that it is perilously close to coming across as a kind of remake or pastiche of the original. But it is not that. It is actually an homage to the original. There are a lot of parallel details and actual "easter eggs," where props make cameos, audio clips from the original movies are sprinkled into the new one. In one of my favorite moments, on Starkiller base we hear a clip from the first movie, "we think they may be splitting up." Some reviewers have made their reviews catalogs of these moments, and consider this excessive, complaining about the "nostalgia overload." But although it is noticeable, I think the producers knew just how much nostalgia would be appreciated, and how much would become annoying, and walked that line very well. The film re-creates the world where Han Solo and Leia Organa will not look out of place. And when Harrison Ford and Carrie Fisher actually appear on screen, the weight of almost 40 years suddenly lands on me, and it's a gut punch. I must have gotten something in my eye.

While Solo is an important character in this film, Leia has very few scenes, and the action centers largely around new characters. The casting is what you might call modern. No one could claim that Rey is not a strong, compelling female character. Daisy Ridley's acting in this movie is very impressive. Without her strong performance, we'd be prone to spend time musing on the oddness of her almost-absent back-story. As it is, we aren't really given a lot of time to meditate on such things, because she keeps very busy, kicking ass and taking names.

John Boyega as Finn is good too, although he doesn't seem, to me, to quite inhabit his character the way Ridley inhabits Rey. And so I find myself spending a little time wondering about the gaps and inconsistencies in his character's back-story. He describes himself as a sanitation worker. If that's true, why is he on the landing craft in the movie's opening scenes, part of the First Order interplanetary SWAT team sent to Jakku to retrieve the MacGuffin? He's supposedly a low-level worker on Starkiller base, but he knows how to disable the shields? He's a stormtrooper, trained since birth to kill, but unable to kill. Has he never been "blooded" before? We're unfortunately reminded that this doesn't actually make a lot of sense. Of course this is true of many elements of the original trilogy. The key to making that kind of thing not matter, for a Star Wars movie, is to keep everything happening so fast that the audience doesn't have time to worry about all that.

The story moves along quickly and we meet one of the most interesting characters, Kylo Ren, played by Adam Driver. Driver plays an adolescent, and puts Hayden Christiansen's portrayal of Anakin to utter shame -- although one senses that much of Christiansen's failure may have been due to Lucas's poor direction of the young actor. Driver is completely compelling on-screen, and his scenes with Ridley are just mesmerizing. I really can't say enough good things about them. I've seen two screenings now, and I would happily see it again, just to watch those two characters interact. It's really impressive.

That's really enough to hang a movie on -- a few really great performances, a few good performances, some terrific scenes, and no scenes that are actually bad. (Howard Hawks famously said that to make a good movie, you needed three good scenes and no bad ones; The Force Awakens exceeds that requirement).

Of course, there are a lot of confusing, unconvincing, and unwieldy things about this film. For example, Rey is much stronger in the ways of the force, and a very powerful fighter, right off the bat. She's grown up on Jakku, apparently spending years alone, and entirely untrained, while in the original trilogy we watched Luke start off with some talent for using the Force, but not much skill, and get trained up like Rocky Balboa. How did this come to pass? Well, it's a mystery we just have to accept for now. Maybe she had a lot of karate classes as a very young child. I maintain that when a movie likes this leaves things unexplained, the audience will do the work for the screenwriters and make it work -- if the audience has decided to side with the movie and help it along. And if they haven't, no amount of rationalization will explain away the inevitable plot holes in a satisfying way. This movie has done such a good job at entertaining the audience, and introducing a compelling character early on, that we as the audience are pretty happy to go along, and willing to make a few allowances and give it the benefit of the doubt. With the prequels, we were bored and full of doubt, for good reasons.

There are a few flaws that I think are worth noting. The movie is just slightly too long. The reawakening of Artoo-Detoo, just after the destruction of the big bad Starkiller base, allowing the plot to continue with a literal deus ex machina -- is just slightly too silly.

What is up with Kylo Ren's helmet, and Captain Phasma's helmet? One of the notable things about the Empire was the extreme precision and cleanliness of the costumes, including the stormtrooper helmets and Darth Vader's helmets. But in the new movie, Ren's helmet is dinged and dented, with chipped paint, and Phasma's helmet is covered in fingerprints. It's not accidental; even the action figures of Kylo Ren have molded-in dents, and there is no way that someone simply forgot to polish Phasma's helmet; such an error would certainly be caught. They were made to look that way deliberately, in stark contrast to the other uniforms and suits of armor. Why is that?

There are some scenes with the Resistance, preparing X-wing fighters, that look like they were literally shot on the site of a freeway overpass; that reminded me of the way J. J. Abrams decided it was a good idea to use a brewery for the engine room of the Enterprise -- an incredibly dumb, unconvincing, revisionist look for the Engineering set. The Imperial wreckage on Jakku -- both Imperial Star Destroyers and the walkers from the invasion of Hoth in Empire -- is nostalgic, but bizarre.

There are some coincidences that feel just a little too coincidental. How did Luke's lightsaber wind up in Maz's basement, in an unlocked trunk, in an otherwise empty room?

Starkiller Base makes very little sense; the physics of it just don't work, in any reasonable universe. The Resistance leaders explain that it sucks up "the sun" -- not "the nearest star" -- in a galaxy with billions of suns, in a film set on multiple planets, around multiple stars, the producers apparently don't trust the audience to understand how stars and planets work; don't confuse them!

But none of this is really a deal-breaker, because the movie moves so fast, and is so willing to break things. Which brings me to the biggest spoiler of all.

The movie kill Han Solo. Yes, they went there. It was at that moment that the film won me over completely. It was a brave move, and it needed to happen. The screenwriters, including Lawrence Kasdan, who worked on Empire, knew very well that if the audience was to take this movie seriously, it would need to show them that it was serious. That's what the death of Han Solo means. Harrison Ford -- who, by the way, is excellent in this film -- has a terrific death. This is also the reason that, for episode IX to work, the screenwriters will have to kill another major character -- most likely, General Leia -- in the first ten minutes.

Given the impressive start to that trilogy, I believe they will do the right thing -- and it will be glorious. And we'll regard the prequels as an unfortunate, non-canon interlude, a mere glitch, in the continuity of the Star Wars story -- and Lucas will continue his slide into irrelevant lunacy.

And meanwhile, as I approach fifty, I still have to wonder. What was the point of Star Wars? Was it ever anything resembling a genuine artistic statement, or was it always a coldly calculated money-grabbing machine, powered by myth, in which Lucas figured out how to monetize the scholarship of Joseph Campbell? Was Star Wars ever actually about anything? Was it "real" art, more than a dizzying whirlwind of entertainment, built on genre tropes and with very little in it that was groundbreaking but the improved technology of movie-making?

Was I simply bamboozled, as a child, into imagining that I was seeing a piece of art, something meaningful? If so, does it matter? Is that dizzying whirlwind of entertainment, blended with a calculated human story arc, really enough? Can real art ever be made out of genre fiction? How about Tolkien? What about smashed-together, postmodern genre fiction? Is it just screenwriting that somehow loses the status of "art?" If I enjoy both Moby Dick and Star Wars, is there something wrong with me?

And, if these distinctions don't matter, and the Disney corporation buys George Lucas's property for four billion dollars, knowing they will turn enormous profits on that investment for decades, and makes us a compelling Star Wars entirely cynically, built literally out of the formulaic building blocks of the original, but it works as well, as wonderfully -- distractingly, entertainingly, wonderfully -- as the original, does that matter? And what does it say about art, and about its audience?

Saturday, November 14, 2015

Working with a Thermistor: a C Programming Example

Recently I worked on a project that needed to monitor temperature using a thermistor. A thermistor is a resistor that measures temperature: the resistance changes depending on how hot it is. They are used in all kinds of electronic devices to monitor temperature and keep components from oveheating.

I searched for a good, simple worked-out example for how to get an accurate reading from the thermistor, but had trouble finding something readable. I am not actually an electrical engineer and have never studied the math behind thermistors formally, but I was able to adapt the formulas from some existing sources, with a little help. I am sharing this example in the hope that it might be useful to someone trying to solve a similar problem. Please note the way I have adapted the general thermistor math to our particular part and circuit; unless you have an identical part and measurement circuit, you will probably not be able to use this example exactly "as-is."

As I understand it, most modern thermistor components are "NTC," which means that they have a "negative temperature coefficient," meaning that their resistance has an inverse relationship to temperature: higher temperature, lower resistance. Thermistors have highly non-linear response, and are usually characterized by the Steinhart-Hart equation. This equation is a general equation that can be parameterized to model the response curve associated with a specific thermistor device. The original form of the equation takes three coefficients, A, B, and C, and describes the relationship between thermistor resistance and temperature in degrees Kelvin (K). It turns out that the three-coefficient form is overkill for a lot of parts and their response curve can be characterized accurately with a single parameter, using a simplified version of the equation This single parameter is called "beta" and so the equation can be called the Beta Parameter Equation.

Reading a thermistor is complicated by the fact that in a typical application we are first using resistance as a measurment of, or proxy for, temperature; that's the basic thing a thermistor does. But in a circuit we don't read resistance directly; instead, we would typically read voltage as a measure of, or proxy for, resistance. To read the resistance from a thermistor we treat it like we would treat a variable resistor, aka potentiometer. We use a voltage divider. This consists of two resistors in series. In our case we place the thermistor after a fixed resistor, and tap the voltage in between. This goes to an ADC - and analog-to-digital converter. I'm going to assume that you already have a reasonably accurate ADC and working code to take a reading from it.

So now I'm going to describe how I took the general thermistor math and adapted it for a specific part and circuit. Our specific thermistor is a Murata NCP18XH103F03RB. So you can Google the vendor and part number and find a datasheet. You need to find out a few things from the datasheet, specifically the nominal resistance at the reference temperature, which is usually 25 degrees Celsius, or 298.15K (or if it is not, note the reference voltage). Also, the datasheet should specify the beta value for your part; in our case, it is 3380.

The beta parameter equation, solved for resistance, reads:

Rt = R0 * e^( -B * ( 1 / T0 - 1 / T ) )

Where Rt is the resistance as a proxy for temperature, e is the mathematical constant e, B is beta, T0 is the reference temperature in K, and T is the measured temperature in degrees Kelvin. We want temperature given resistance, so we can solve it for temperature, like so:

T = B / ln( R / ( R0 * e^( -B / T0 ) ) )

Plugging in our R0 = 10,000 ohms, B = 3380, and T0 = 298.15 K we get:

t = 10000 * e^( -3380 * ( 1 / 298.15 - 1 / T ) )

or

T = 3380 / ln( R / ( 10000 * e^( -3380 / 298.15 ) ) )

Now, we need to have something to plug in for R, given the fact that we're reading a voltage from a voltage divider. In our case, the fixed resistor in our voltage divider has the same resistance value in ohms as the nominal resistance for our thermistor at 25 C, 10 kohms (10,000 ohms). Our voltage going into the voltage divider is 2.5V. The standard formula for a voltage divider like this, arranged with the fixed resistor first in the series, before the thermistor, is:

V = 2.5 * ( R / ( 10000 + R ) )

If your thermistor comes before the fixed resistor, you will want to swap the two R values values (see the Wikipedia article on voltage dividers I mentioned above). To get resistance given voltage, we can solve the above for R:

R = 20000 * v / ( 5 - 2 * v )

Now we've got a formula that we can use to convert a voltage reading to a thermistor resistance reading R. We can actually plug the right hand side of that right into our beta parameter equation from above, replacing R:

T = 3380 / ln( ( 20000 * v / ( 5 - 2 * v ) ) / ( 10000 * e^( -3380 / 298.15 ) ) )

That looks kind of monstrous; it really seems like this ought to be simpler than that. But Wolfram Alpha could simplify it when my own algebra skills gave out. You can just go to the Wolfram Alpha site and paste in that equation, being careful to get the parentheses in the right place. You will want to change the T to a t so that Wolfram Alpha interprets it as a variable, rather than Tesla units. Here's the result. Note that Wolfram Alpha has provided a nicely simplified version of the equation, perfect for our needs:

t = 3380 / ln( 167665 * v / ( 5 - 2 * v ) )

That equation describes temperature, in K, as a function of the measured voltage from our specific thermistor and voltage divider circuit. Again, keep in mind that unless you have an identical part and circuit, you will not be able to use this formula as-is. Testing against a thermocouple and hand-held infrared thermometer suggests, so far, that our temperature readings seemed to be accurate to within a degree C. We have not tested it with extremely high or low temperatures yet, but I expect it to be reasonably accurate; for this application, which involves setting fan speed and determining if we need to shut down components, we don't need a high degree of accuracy.

Finally, remember that the results of this formula are in degrees Kelvin. A C programming language expression for converting degrees Kelvin to degrees Celsius is simply:

k - 273.15F

where k is a floating-point value representing degrees Kelvin. Similarly, you can convert to degrees Fahrenheit like so:

k * 9.0F / 5.0F - 459.67F

and the C expression to implement our voltage-to-temperature function is:

3380.0F / log( ( 167665.0F * v ) / ( 5.0F - ( 2.0F * v ) ) )

where v is a floating-point value representing the voltage from our voltage divider, and log is the C programming language's natural logarithm function, part of the C standard library of math functions.

Please take care when using expressions like this. Note that it contains floating-point division operations. You must check to make sure that the values you are dividing by are non-zero! If not, the result will be a floating-point "not-a-number" value (NaN). Depending on your platform, this may happen silently, without any sort of runtime error, and the result will then "poison" any downstream calculations carried out with this value.

I hope this has been helpful. Please leave a comment (note: comments are moderated, so you will not see them appear immediately) if you were able to adapt this approach to your project! If you have a question, you can leave a comment too, although keep in mind that I'm not actually an electrical engineer and so would be better at answering programming questions than circuit design questions. Happy measuring!

Thursday, February 05, 2015

A Deep Dive: the Velocity Manufacturing Simulation

In 1989 I graduated from the College of Wooster and then spent a year as an intern with Academic Computing Services there, writing newsletters and little software tools. In the summer of 1990 I moved to Ann Arbor, without a clear idea what I was going to do next.

I worked for a short while with the Department of Anthropology, but by the end of 1990, I had found a job with the Office of Instructional Technology.

OIT was sort of the University's answer to the MIT Media Lab. It was an organization where instructional designers, programmers, and faculty members could work together on projects to bring technology into classrooms. It was a pretty remarkable workplace, and although it is long gone, I am truly grateful for the varied experiences I had there. It was the early days of computer multimedia, a sort of wild west of platforms and tools, and I learned a lot.

In January of 1993 my girlfriend and her parents visited my two workplaces, OIT headquarters and the Instructional Technology Lab, a site in the Chemistry building. I handed my girlfriend a video camera and proceeded to give a very boring little talk to her, and her extremely patient parents. Wow, I was a geek. I'd like to think my social skills and ability to make eye contact are a lot better now, but I probably haven't changed as much as I imagine that I have. I'm an extraverted geek now: when I am having a conversation with you, I can stare at your shoes.

I have carried the original analog Hi-8 videocassette around through many moves, and life changes, and only today figured out a good way to get it into my computer -- after giving the camcorder heads a very thorough cleaning. I thought the tape was pretty much a lost cause, and was going to try working with my last-ditch backup, a dub to VHS tape, but I'm pleased to learn that the video is still playable, and pleased that I could finally get this made, such as it is.



This project, the Velocity Manufacturing Simulation, was written in Visual BASIC, long before it became VB.NET. I remember that it involved a fair amount of code, although I don't have the source to look at. I remember painstakingly writing code for GUI elements like the animated disclosure triangles. There was some kind of custom controls library we bought separately; the details escape me. There was some kind of ODBC (maybe?) database plug-in that I can barely recall; I think Pete did most of the work on that part. Pete wrote parts of it, and I wrote parts of it. Now it seems almost laughably primitive, but you'll just have to take my word for it that back in the day it seemed pretty cool. It won an award. As far as I know, this is the only video footage of the project.

The code is 147 years old in Internet years. It was almost half my lifetime ago. But at the same time it seems like I just left that office, and somehow if I could figure out where it was, I could still go back and find everyone there in the conference room having lunch, and after lunch settle back into my old office with the vintage, antique computers.

This was only one of several projects I worked on while I worked at OIT. I have some other bits of video for a few of them, but not all. I will get clips up for at least one more. I wish there was more tape, and better tape, even if the only one nostalgic about these projects is me.

Perhaps "enjoy" is the wrong word, but take a moment to remember what instructional multimedia was like, a few months before a group called NCSA released a program called Mosaic and the world started to hear about this exciting new thing called the World Wide Web... but grandpa's tired, kids, and that's a story for a different day.