Programming


C++ stuff

Coming from C to C++ there are higher level types to do what is needed. Use them instead.

They are:

Arrays -> Vectors
char arrays -> String
bitset...


Notes:
cin only reads up to the next whitespace.
getline reads a line
Store string::size in the string::size_type object....

Authentication

For iPhone, in app google authentication:

For Web:

To Do:
Use rails
Use google apps account for domain
Use apps calendar for scheduling
Use apps contacts for address book
+ set up phone to use apps contacts
Use apps mail to send/receive emails
Link out as much as possible to web

To Do v2:
Integrate above features properly
Add offline mode:
  • Set token age to be a day to cover travelling times. Log in once in the morning and you have auth access for the day
  • Give items a mode, auth and unauth
  • If no data, can view and create unauth items, cannot view authed items
  • unauth still requires propper username to auth against later


AVR and SD cards

Working with SD cards are annoying but comparatively easy and cheap.

I bought an Arduino board and an SD card shield.
There is a local electronics place which are really good.

Arduino board (I should have gotten that one :(, instead I got a Diecimila clone).
SD Shield (Warning, no documentation or anything is provided by the makers. I recommend any other one or you can make the interface yourself.)
SD card. Note that the Shield is not SDHC so I have a standard 2GB card.

Below is the code for running the SD card and the Arduino board in general. However it may not work immediately as I can't guarantee that my board is 100% Arduino compatible etc. But my processor is an ATMega328P.

I'm using the SPI interface to work with my SD card. The code is fairly correctish to version 3 of the SD specifications. The simple version of the SD specifications are available at the SD Association.
Here is the code to use the SD card. It is quite long but its all there. I don't check for card type or voltage range or anything.
There are things to note. The SD card runs at 3.3V, it cannot be SDHC, thats about it. 
Big note about blow. SS is the same as CS... SS is used when talking about SPI stuff and CS is used when talking about the SD card. 
Also I sort of skimped on the header files below but you can sort that out yourself (and the appropriate include lines).

spi.c:

#include <avr/io.h>

// specific definitions to set the speed
#define SLOW 0xF0
#define FAST 0xF1

// port definitions for the SD card as it is on my board
#define PORT_SPI PORTB // values port b 
#define DDR_SPI DDRB // direction port b 
#define DD_SS   DDB2 // pin 10 also known as CS
#define DD_MOSI DDB3 // pin 11
#define DD_MISO DDB4 // pin 12
#define DD_SCK  DDB5 // pin 13
#define DD_SDIN DDB1

// silly little macro
#define CLEAR_SS()      CLEAR_BIT( PORT_SPI, DD_SS )
#define SET_SS()        SET_BIT( PORT_SPI, DD_SS )

uint8_t SPI_INIT = 0;
void SPI_init( void );

// Set the speed of the SPI interface
void SPI_set_speed( uint8_t speed )
{
	if( SPI_INIT == 0 ) SPI_init(); // if we aren't initialised then do so

	if ( speed == SLOW ) 
	{
		// setting 011 = fosc/128 = 16MHz/128 = 125KHz
		SET_BIT( SPCR, SPR0 );
		SET_BIT( SPCR, SPR1 );
		CLEAR_BIT( SPSR, SPI2X );
	}
	else if ( speed == FAST ) 
	{
		// Setting 100 = fosc/2 = 16MHz/2 = 8MHz
		CLEAR_BIT( SPCR, SPR0 );
		CLEAR_BIT( SPCR, SPR1 );
		SET_BIT( SPSR, SPI2X );
	}
}

// Don't call this directly. All the other functions will
void SPI_init( void )
{
	SPI_INIT = 1; // we are initialised, so... yay!

	// Set SS high. SS low activates a transfer.
	// Set SS, MOSI and SCK as output pins
	// Set MISO as input
	SET_BIT( PORT_SPI, DD_SS ); 
	SET_BIT( DDR_SPI, DD_SS ); 
	CLEAR_BIT( DDR_SPI, DD_MISO ); 
	SET_BIT( DDR_SPI, DD_MOSI );
	SET_BIT( DDR_SPI, DD_SCK );

	// Note, SD cards need to do less than 400KHz during SD card initialisation
	// 16MHz / 32 = 500KHz
	// 16MHz / 64 = 250KHz
	// 16MHz / 128 = 125KHz
	// We could use 64 or 128. I don't know any advantage of either
	SPI_set_speed( SLOW );

	// Set this to be the Master and enable SPI
	SET_BIT( SPCR, MSTR );
	SET_BIT( SPCR, SPE );
}

// This is the core SPI data transfer. To get data we must send data as well!
uint8_t SPI_send_get( uint8_t data )
{
	if( SPI_INIT == 0 ) SPI_init(); // initialise SPI. Means we don't have to call init elsewhere
	
	SPDR = data; // put data in register to send
	while ( ! ( SPSR & _BV( SPIF ) ) ); // wait for transmission to complete
	return SPDR; // read received data
}

// load data into the register to send and wait for the transmit to complete
void SPI_send( uint8_t data )
{
	SPI_send_get( data );
}

// 0xFF is a dummy value that the SD card ignores
uint8_t SPI_get( void )
{
	return SPI_send_get( 0xFF ); // dummy value to send
}

sd.c:
This is a bit messy and is really an exaggeration because I was trying to keep to the spec.
There is no main code here. Simply create a uint8_t buffer[512], read in some data at some location, change it and write it back.

#define BUSY_STATE 0

// Definitions for SD card commands
#define CMD0 0
#define CMD8 8		// not valid for 1.x cards I think
#define CMD9 9		// Card to respond with CSD 
#define CMD16 16	// Set block length for next command
#define CMD17 17	// Read single block
#define CMD24 24	// Send a block to sd card
#define CMD55 55	// Next command is ACMD

// Enter initialising, 0x80 is for the code to recognise it as ACMD so it knows to send the CMD55 before
#define ACMD41 0x80 | 41

// SD card sends this byte as an identifier
#define START_BLOCK 0xFE

// Used to simplify a match for isValidResponse function
#define DIRECT_MATCH	0x00
#define BITS_SET	0x01
#define BITS_NOTSET	0x02
#define ANY_GOOD_BYTE	0x03

// defining ACMDs to have MSB set.
// A valid CMD for SD card has MSB unset so we are stealing it :D
// Make sure you use makeCMD!

#define isACMD( data ) data & 0x80
#define makeCMD( data ) data & 0x7F

#define VALID_R_BYTE BITS_NOTSET << 8 | 0x80
#define READY_STATE DIRECT_MATCH << 8 | 0
#define IDLE_STATE DIRECT_MATCH << 8  | 1
#define ANY_BYTE ANY_GOOD_BYTE << 8 | 2

uint8_t SD_INIT = 0;

/*
Data Read:
Using single blocks: CMD17
	Send CMD16 to set the amount of data to get
	Send CMD17 
	Card will respond with a token
	Card will send the data with 2 bytes of CRC or a data error token

Data Write:
Using single blocks: CMD24
	Send CMD24
	Card will response with a token
	Send Start token
	Send Data
	Card sends data-response token 
	Card sends a stream of busy tokens while saving data
	Should send CMD13 to see if saving went ok
*/

void sd_init( void )
{
	SD_INIT = 1;
	uint8_t response;

	// We need to send at least 74 clicks 
	// SS must be high during these 74 clicks
	int i = 0;
	for ( ; i < 10; i++ ) SPI_send( 0xFF );

	// send required init commands
	CLEAR_SS();

	// Send Command 0, and we want the card to get into an idle state so it tries again a few times
	response = send_command_r( CMD0, 0, IDLE_STATE );
	if ( !isValidResponse( response, IDLE_STATE ) ) // If it fails we can't use this card as it is
	{
		SD_INIT = 0;
		return;
	}

	// The card is in an idle state now!

	// Send CMD8, if we get back a valid byte we don't know continue on.
	// It means a pre version 3 card.
	response = send_command_r( CMD8, 0x000001AA, VALID_R_BYTE );
	if ( isValidResponse( response, VALID_R_BYTE ) ) // SD card knows CMD8
	{
		uint8_t R7[4];
		R7[0] = SPI_get(); // command version
		R7[1] = SPI_get();
		R7[2] = SPI_get();
		R7[3] = SPI_get();
	}
	else  // I'm going to require a current SD card because the new ones support single byte writes
	{
		SD_INIT = 0;
		return;
	}

	// Send ACMD41 to get out of an idle state
	response = send_command_r( ACMD41, 0, READY_STATE );
	if ( !isValidResponse( response, READY_STATE ) )
	{
	        // we sent a reasonable amount of ACMD41s and didn't get back a ready state
		SD_INIT = 0;
		return;
	}

	SET_SS(); // we are done sending and receiving data

	SPI_set_speed( FAST );
	// after we want to jump up to a nice speed
	// we should now be ready to read write to the card

}

// this function eats up a bunch of data after a read command is sent until we see the start token 0xFE
uint8_t get_data_start()
{
	uint16_t timeout = 0xFFFF;
	uint8_t response = 0xFF;
	while ( --timeout > 0 )
	{
		response = SPI_get();
		if ( response == 0xFE ) break;
	}
	return response;
}

// Send a cmd over the SPI lines
// Wait until there is a valid response
uint8_t send_command( uint8_t data, uint32_t args )
{
        if ( SD_INIT == 0) sd_init();
        // send a command and print out what we get as a response
        // I think that we send a command and wait for the response to not be FF
        // the return that response and let somewhere else handle it.
        // each command is different so maybe have a different function for each command
        // I run with CRC disabled so It is set for the two instructions that require CRC.

        SPI_send( 0x40 | data );
        SPI_send( args >> 24  );
        SPI_send( args >> 16  );
        SPI_send( args >> 8  );
        SPI_send( args );
        switch( data )
        {
                case CMD8: // only valid for 0x000001AA
                        SPI_send( 0x87 );
                        break;
                default:
                        SPI_send( 0x95 ); // Its disabled except for the CMD0 so we can leave it as this
                        break;
        }

        // Waiting for a valid packet to come back
        uint16_t response = 0xFF;
        uint16_t timeout = 0xFFFF;
        while( response == 0xFF && --timeout > 0 )
        {
                response = SPI_get();
        }

        // if response == 0xFF there has been a timeout
        return response;
}

// this function sends a command and expects a specific response
// to make this simpler there is the isValidResponse which does the matching of the
// required response data
uint8_t send_command_r( uint8_t data, uint32_t args, uint16_t required_response )
{
	uint8_t response;
	uint16_t timeout = 0xFFFF;
	// we want a specific response. If we don't get it send the command again
	// do while loop saves a duplicate line :P
	do 
	{
		// if this is ACMD we need to send CMD55 first.
		// CMD55 sends back R1, but we are going to ignore it...
		if ( isACMD( data ) )
		{
			send_command( CMD55, 0 );
		}
		// makeCMD strips out the ACMD identifier if it has one
		response = send_command( makeCMD( data ), args );
	}
	while ( --timeout > 0 && !isValidResponse( response, required_response ) );

	return response;
}

// helper function for simplifying a while loop
uint8_t isValidResponse( uint8_t data, uint16_t validtype )
{
	// valid type is two bytes, the high byte is the type of match to do
	// the low byte is the valid data
	uint8_t match_type = validtype >> 8;
	uint8_t valid_data = validtype & 0xFF; // I probably don't need to mask but anyways
	switch( match_type )
	{
		case DIRECT_MATCH:	return data == validtype;
		case BITS_SET:		return data & valid_data;
		case BITS_NOTSET:	return !( data & valid_data );  
		case ANY_GOOD_BYTE:	return !( data == 0xFF );
		default:		return 0;
	}

}

// read 1 block (512 bytes) of data into the buffer
// start_loc must be block aligned.
uint8_t read_block( unsigned int start_loc, uint8_t *buffer )
{
        if ( start_loc % 512 != 0 ){
                return 0; // need to be at the start of a block
        }
        uint8_t response;
        uint32_t length = 512;

        // Set that we are sending data
        CLEAR_SS();

        // Set the block length to be 512bytes
        send_command_r( CMD16, length, VALID_R_BYTE );

        // send the get data command
        send_command_r( CMD17, start_loc, VALID_R_BYTE );
        // eat up all empty bytes
        get_data_start( );

        // now we get the data
        if ( buffer ) // don't bother if we have a null pointer
        {
                uint32_t x = 0;
                while( x < length ) buffer[x++] = SPI_get();

                response = SPI_get(); // crc byte
                response = SPI_get(); // crc byte
        }
        SET_SS();
        return 1;
}

// similar to the read block
// writes the 512 bytes of buffer to the sd card at start_loc
// this function waits for the SD card to write the data
uint8_t write_block( unsigned int start_loc, uint8_t *buffer )
{
        if ( start_loc % 512 != 0 ){
                return 0; // need to be at the start of a block
        }
        uint8_t response = 0xFF;
        uint32_t length = 512;

        CLEAR_SS();
        // Set the block length to be 512
        send_command_r( CMD16, length, VALID_R_BYTE );

        // send the write data command
        send_command_r( CMD24, start_loc, VALID_R_BYTE );
        
        // we send a start token then the data
        SPI_send( START_BLOCK );

        // now send data
        uint32_t x = 0;
        while( x < length ) SPI_send( buffer[x++] );

        SPI_send( 0x95 ); // crc byte
        SPI_send( 0x95 ); // crc byte

        uint16_t timeout = 0xFFFF;
        while( --timeout > 0 )
        {
                response = SPI_get();

                // first bit must be 1  and 4th bit must be 0
                //if ( !(response & 0x1) && response & 0x10 )
                if ( response != 0xFF )
                        break;
        }

        timeout = 0xFFFF;
        while ( --timeout > 0 && SPI_get() != BUSY_STATE ); // eat up busy bytes

        // I should check send status ( cmd13 )
        SET_SS();
        if ( (response & 0xF) == 5 ) 
        {
                // Data accepted
                return 1;
        }
        else if ( (response & 0xF) == 11 ) 
        {
                // Data CRC Error
                return 0;
        }
        else if ( (response & 0xF) == 13 ) 
        {
                // Data Write error
                return 0;
        }
        else 
        {
                // Not sure what the status is
                return 0;
        }

        return 0;
}

AVR, Arduino, USB Serial and printf

I have found out how I can use input/output to my Arduino board. Well output, I suppose input is similar.

I'm programming my Arduino board in C and I'm not using any of the Arduino libraries or anything like that. Standard C with standard libs and whatnot.

To get output working over the Serial, I need to get USART working as that is how the Arduino board is connected to the USB.
Getting USART working is probably the easiest thing ever. The datasheet for my processor has some C examples.
I just had a look and my code is pretty much copied from the datasheet.

Some simple code:

// F_CPU is passed down from the Makefile
#define BAUD 9600
#define MYUBRR F_CPU/16/BAUD-1
#include <avr/io.h>
#include <stdio.h>
// setup USART
void USART_init( void )
{
        unsigned int ubrr = MYUBRR; // from datasheet
// Disable USART so we can configure it
UCSR0B = 0; // turn everything off
// Set baud rate defined by MYUBRR
UBRR0H = ( uint8_t ) ( ubrr >> 8 );
UBRR0L = ( uint8_t ) ( ubrr );
// Turn off double speed
UCSR0A &= ~(1 << U2X0);
// Enable TX/RX
UCSR0B |= 1 << RXEN0 | 1 << TXEN0;
// Asynchronous, No Parity, 2 stop bits, 8 databits
UCSR0C = 1 << USBS0 | 1 << UCSZ01 | 1 << UCSZ00;
}
// send a byte 
// Only works on 8bits or less frame size
// The return and parameter types are required by printf
int USART_transmit( char data, FILE *stream )
{
        // simple copy from page 185 of the data sheet
        // Wait for empty transmit buffer 
        while ( !( UCSR0A & (1<<UDRE0)) );
        // Put data into buffer, sends the data 
        UDR0 = data;
}
// this is how we tell printf to output to USART (can be replace to output to lcd or something)
// Also need the stdout line in main
// Set to write because I don't handle input. and I probably would do input from another function
FILE usart_print = FDEV_SETUP_STREAM( USART_transmit, NULL, _FDEV_SETUP_WRITE );
int main( void )
{
stdout = &usart_print; // this is also needed so printf works
printf( "You should be able to see this over the serial port\n\r", x );
for(;;){ // infinite loop 
}
        return 0;
}

Note that USART_transmit( char data, FILE * stream ) is like this so we can use printf and this way we don't have to worry about handling it ourselves.
Also that on my machine I need to use new line carriage return otherwise my terminal program has issues on newlines.

Compile that and flash the board. To connect to the serial in terminal use the screen program. NOTE: I miss putty for this stuff. screen is nice but annoying. Sigh. To quit screen you must use control + A then control + \. If you quit another way then screen stays in the background and keeps hold of the serial device. Meaning you can't reprogram and you can't reconnect until the screen process is killed.

Run:
screen /dev/tty.usbserial-XXXXX # whatever the device filename is

You should see the output from your board when it boots up.
NOTE my board restarts when connecting over serial as part of the auto reboot. I'm not sure if the Arduinos do that or if it happens at other speeds etc.

AVR and Arduino and not C++

I bought:
  • Arduino clone (I wish I bought the real thing btw) ATMEL AVR ATMega328p.
  • SD card shield. Not SDHC compatible but I can live.
  • 2GB SD card (not SDHC).

I bought the stuff locally in Canberra from Australian Robotics. They are really well priced and I spent over $50 so it only cost $3.50 for express postage. My order was waiting for me at home the next day. They are awesome.

Whats really nice about the Arduino is that it s a really cheap and easy way to get into programming microprocessors. The board comes pre made with easy access to the pins, the clock installed, all resisters and such installed. These boards can be made by hand I think, you can buy kits or what not. 
This biggest advantages about this board is the USB port (which you can power the board off) and the Arduino bootloader. You do not need another chip or doodad to program these boards which is awesome.

Back when I started I spent over $200 on microprocessor stuff. Now I can buy all that in the Arduino form for about $20. Awesome.

However, Arduino have made a big effort to make the board really easy to use and make their own software to program it. Their software allows you to program in C++ really easily with a whole bunch of useful examples and code and libraries and everything. Its really nice. Until you realise you need do stuff at 2microseconds...

The reason I got the Arduino is because I want to make a floppy drive replacement device for my Amiga 500. I have a whole bunch of games which instruct you to backup the disk and also requires 1 or 2 disks to save data 2. I don't have any disks and even if I did I can't format them right now because I don't have the Workbench disks (OS disks).

So I have this awesome Arduino but I don't want to use their software. Well you at least need to install their software AND the FTDI drivers to talk to the board over USB.
I also use the Arduino sample code to make sure things on my board are working. Also you can download the Arduino source and look at their libraries like the Serial and SD objects and how they do delays and stuff. Really nice if you get stuck.
If you hold down shift while uploading a sketch to the board it will output a whole bunch of info in the console area of that window. From this info I was able to figure out how to send compiled code to the board.
Note I'm doing this on Mac OS X. However much of this stuff is probably available for windows and Linux. Alternatively you can use the info from the Arduio software to figure it out. On Windows you probably could use AVR Studio to compile the code and use the command from the Arduino software to send the data to the board if AVR Studio can't do it.

Note about the bootloader:
The bootloader seems to handle all the programming to the board which is good. We do not want to ever overwrite this without very good reason. The Arduino bootloader is really awesome since we can program the board over USB really really easily. Also from what I can tell, the bootloader has been protected from being overwritten by default using the fuses. We can pretty much ignore the bootloader and let it do it's magic. Just be aware that it is there, it is awesome and you always want it.

On my Mac I installed the Arduino software and a really nice package of stuff CrossPack.
I'm not sure if CrossPack has more stuff in it than the Arduino software but it has avr-gcc and avrdude. gcc is the compiler and avrdude is used to send the compiled program to the board. (gcc and avrdude also come with the Arduino software).

I ran the avr-project command which gave me a very basic Xcode project and a separate basic makefile and main.c file.
I don't bother with Xcode, I use vim for all my work so I can't comment on how to get that working.
The great thing about having the Makefile for you is that it is really easy to program C and C++ and get it all compiled and sent to the board. I assume that it is also easy to compile assembler. 
Oh also if you want the speed of assembler but you like C, you can program everything in C then compile it and use avr-objdump to print out the assembler which you can fiddle with and recompile if you don't like how the compiler optimises your code. This is really helpful also if you want really simple assembler for something you know how to do easily in C.

Compiling and Flashing
To be able to compile and to get make to work you need to fix the Makefile for your board.
My board has an atmega328p processor. You can get info for boards from the Arduino source (boards.txt file I think).
First I comment out all references to FUSES. I don't know much about them other than they control how the processor works, protect the bootloader and a bunch of other stuff. I don't know about them and so I don't want to touch them, everything seems to work fine without fiddling with them. The default makefile is for an ATMega8 which may not have the same fuse settings as my processor. 
Also you need to update the OBJECTS section with the new files you add to your project otherwise they won't be compiled.

The important lines to change are:
DEVICE     = atmega328p
CLOCK      = 16000000
PROGRAMMER = -D -b57600 -c arduino -P /dev/tty.usbserial-A400fXlf 

Not sure what device you have? run avrdude -p ? to list avrdude supported processors. 
My board is configured with a 16MHz crystal.
PROGRAMMER was the hard one to figure out:
  • -D: Disable auto erase for flash memory... I don't know if thats a good or bad idea.
  • -b 57600: The baudrate needed to connect to the board via the USB port. I'm pretty sure this is set as part of the bootloader and FTDI set up on the board.
  • -c arduino: The programmer type. Usually you need a programmer to flash your processor. However, the Arduino bootloader is awesome and can program itself.
  • -P /dev/tty.usbserial-XXXXX: This is the device handle used to talk to the board. This is also the device to connect to when you want to send/receive stuff over serial.

With all that set up and some code in main.c you should be able to run make to compile and if everything goes well run make flash to program your board.

Pages

Subscribe to RSS - programming