WWW.DUMAIS.IO
ARTICLES
OVERLAY NETWORKS WITH MY SDN CONTROLLERSIMPLE LEARNING SWITCH WITH OPENFLOWINSTALLING KUBERNETES MANUALLYWRITING A HYPERVISOR WITH INTEL VT-X CREATING YOUR OWN LINUX CONTAINERSVIRTIO DRIVER IMPLEMENTATIONNETWORKING IN MY OSESP8266 BASED IRRIGATION CONTROLLERLED STRIP CONTROLLER USING ESP8266.OPENVSWITCH ON SLACKWARESHA256 ASSEMBLY IMPLEMENTATIONPROCESS CONTEXT ID AND THE TLBTHREAD MANAGEMENT IN MY HOBBY OSENABLING MULTI-PROCESSORS IN MY HOBBY OSNEW HOME AUTOMATION SYSTEMINSTALLING AND USING DOCKER ON SLACKWARESYSTEM ON A CHIP EMULATORUSING JSSIP AND ASTERISK TO MAKE A WEBPHONEC++ WEBSOCKET SERVERSIP ATTACK BANNINGBLOCK CACHING AND WRITEBACKBEAGLEBONE BLACK BARE METAL DEVELOPEMENTARM BARE METAL DEVELOPMENTUSING EPOLLMEMORY PAGINGIMPLEMENTING HTTP DIGEST AUTHENTICATIONSTACK FRAME AND THE RED ZONE (X86_64)AVX/SSE AND CONTEXT SWITCHINGHOW TO ANSWER A QUESTION THE SMART WAY.REALTEK 8139 NETWORK CARD DRIVERREST INTERFACE ENGINECISCO 1760 AS AN FXS GATEWAYHOME AUTOMATION SYSTEMEZFLORA IRRIGATION SYSTEMSUMP PUMP MONITORINGBUILDING A HOSTED MAILSERVER SERVICEI AM NOW HOSTING MY OWN DNS AND MAIL SERVERS ON AMAZON EC2DEPLOYING A LAYER3 SWITCH ON MY NETWORKACD SERVER WITH RESIPROCATEC++ JSON LIBRARYIMPLEMENTING YOUR OWN MUTEX WITH CMPXCHGWAKEUPCALL SERVER USING RESIPROCATEFFT ON AMD64CLONING A HARD DRIVECONFIGURING AND USING KVM-QEMUUSING COUCHDBINSTALLING COUCHDB ON SLACKWARENGW100 MY OS AND EDXS/LSENGW100 - MY OSASTERISK FILTER APPLICATIONCISCO ROUTER CONFIGURATIONAASTRA 411 XML APPLICATIONSPA941 PHONEBOOKSPEEDTOUCH 780 DOCUMENTATIONAASTRA CONTACT LIST XML APPLICATIONAVR32 OS FOR NGW100ASTERISK SOUND INJECTION APPLICATIONNGW100 - DIFFERENT PROBLEMS AND SOLUTIONSAASTRA PRIME RATE XML APPLICATIONSPEEDTOUCH 780 CONFIGURATIONUSING COUCHDB WITH PHPAVR32 ASSEMBLY TIPAP7000 AND NGW100 ARCHITECTUREAASTRA WEATHER XML APPLICATIONNGW100 - GETTING STARTEDAASTRA ALI XML APPLICATION

REALTEK 8139 NETWORK CARD DRIVER

2013-12-03

While building my homebrew OS, I go to the point where I needed a netcard driver. I run my os in QEMU, which provides a RealTek 8139 netcard. The specs for that card are very easy to find.

Before I continue, you should know that when the datasheet specifies a register that is 2 bytes long (like ISR), it is important to read it as a 16bit word even if all you need is the first 8bit. I was reading ISR with "inb" and couldn't make my software work event if all I needed was the first byte. Changing "inb" for "inw" worked. The datasheet indicates that some registers need to be read or written as words or dwords even if it looks like they could be accessed as bytes.

Initializing

  • Enable the card: OUTPORTB(0,iobase+0x52);
  • Reset the card:
    You need to write the "reset" bit in register 0x37, and then wait until that bit gets cleared unsigned char v=0x10; OUTPORTB(v,iobase+0x37); while ((v&0x10)!=0) INPORTB(v,iobase+0x37);
  • enable TX and RX interrupts: OUTPORTB(0b101, iobase+0x3C); There are other interrupts in register 0x3C that can be interesting but I just need TOK and ROK for now.
  • enable 100mbps full duplex: OUTPORTB(0b00100001, iobase+0x63)
  • Set the Receive Configuration Register (RCR):
    OUTPORTL(0x8F, iobase+0x44);
    Looking at the datasheet, you can see what those bits mean. Bascically what we did is:
    • set promiscuous mode
    • accept frames for our MAC address
    • accept frames for out multicast address
    • accept broadcasted frames
    • Do not accept runts and erroneous frames
    • set the RX buffer size to 8k
    • disable WRAP. This means that is a frame is received and we are near the end of the RX buffer, the card will continue copying data after the buffer. We are basically allowing buffer overflow here. so for this reason, we need to give extra space to our buffer. I chose to use a 10k buffer just to be sure
  • Set the RX buffer address. The details of this buffer will be explained in the next section. For now, let's just reserve a buffer of 34k and tell the card about it: OUTPORTL(buf_addr, iobase+0x30)

    Warning: The addresses for TX and RX buffers must be physical addresses. Not virtual addresses

  • Set the Transmit Configuration Register (TCR): The default values after reset are fine. So I'm not touching that register.
  • Set the tx descriptors for now, I won't go in the details of those buffers, this will be explained in the next section all you need to know right now is that you need 4 2k buffers and tell the card about them
    OUTPORTL(buf_addr_desc0, iobase+0x20);
    OUTPORTL(buf_addr_desc1, iobase+0x24);
    OUTPORTL(buf_addr_desc2, iobase+0x28);
    OUTPORTL(buf_addr_desc3, iobase+0x2C);
  • enable TX and RX: OUTPORTB(0b00001100,iobase+0x37);

This is my init code. Note that there is some PCI stuff in there that I don't describe. I am assuming that you have a PCI driver written at this point

void initrtl8139() { unsigned int templ; unsigned short tempw; unsigned long i; unsigned long tempq; deviceAddress = pci_getDevice(0x10EC,0x8139); // vendor, device. Realtek 8139 if (deviceAddress == 0xFFFFFFFF) { pf("No network card found\r\n"); return; } for (i=0;i<6;i++) { unsigned int m = pci_getBar(deviceAddress,i); if (m==0) continue; if (m&1) { iobase = m & 0xFFFC; } else { memoryAddress = m & 0xFFFFFFF0; } } irq = pci_getIRQ(deviceAddress); registerIRQ(&handler,irq); pci_enableBusMastering(deviceAddress); // Activate card OUTPORTB(0,iobase+0x52); // reset unsigned char v=0x10; OUTPORTB(v,iobase+0x37); while ((v&0x10)!=0) { INPORTB(v,iobase+0x37); } INPORTL(templ,iobase+4); tempq = templ; tempq = tempq <<32; INPORTL(templ,iobase); tempq |= templ; macAddress = tempq; } void rtl8139_start() { // Enable TX and RX: OUTPORTB(0b00001100,iobase+0x37); // Set the Receive Configuration Register (RCR) OUTPORTL(0x8F, iobase+0x44); // set receive buffer address // We need to uses physical addresses for the RX and TX buffers. In our case, we are fine since // we are using identity mapping with virtual memory. OUTPORTL((unsigned char*)&rxbuf[0], iobase+0x30); // this is a 10k buffer // set TX descriptors OUTPORTL((unsigned char*)&txbuf[0][0], iobase+0x20); // 2k alligned buffers OUTPORTL((unsigned char*)&txbuf[1][0], iobase+0x24); OUTPORTL((unsigned char*)&txbuf[2][0], iobase+0x28); OUTPORTL((unsigned char*)&txbuf[3][0], iobase+0x2C); // enable Full duplex 100mpbs OUTPORTB(0b00100001, iobase+0x63); //enable TX and RX interrupts: OUTPORTW(0b101, iobase+0x3C); }

Receiving

Since we have enabled the ROK and TOK interrupts, we will receive and interrupt when a new frame arrives. So from my interrupt handler I check the ISR register to know if I got a TOK or ROK. if ROK, then proceed with getting the frame. First, some definitions:

  • CAPR: This register holds the address within the RX buffer where the driver should read the next frame. This register must be incremented by the driver when a frame is read. The netcard will check that register to determine if a buffer overrun is occuring.
  • packet header: This is a 4bytes field that is found at the begining of the frame. The first word is a bitfield indicating if the frame is OK, if it was received as part of multicast ect. More information can be found in section 5.1 of the datasheet. The following 2 bytes indicate the size of the frame

This is what I do:

  • 1) Trigger on interrupt: Since interrupts have been enabled, IRQ will have been raised. So this will be done from the handler. We need to check TOK in the ISR register
  • 2) Get position of frame within the RX buffer by reading CAPR
  • 3) Get size of data: 2nd 16bit word from begining of buffer (CAPR+2)
  • 4) copy the frame: address starts at rx_buffer_base+CAPR
  • 5) Update CAPR: CAPR=((rxBufIndex+size+4+3)&0xFFFC)-0x10 We are adding 4 to take into account the header size and the +3&0xFFFC is to align on a 4bytes boundary. I have no idea why we need to substract 0x10 from there. Note that you should keep track of rxBufIndex separately. I.e: do not update it with CAPR everytime.
  • 6) Check BUFE bit in CMD. if set, go back to step 2
  • 7) write 1 to ROK in the ISR register

The receiving function:

unsigned long rtl8139_receive(unsigned char** buffer) { if (readIndex != writeIndex) { unsigned short size; unsigned short i; unsigned char* p = rxBuffers[readIndex]; size = p[2] | (p[3]<<8); if (!(p[0]&1)) return 0; // PacketHeader.ROK *buffer = (char*)&p[4]; // skip header readIndex = (readIndex+1) & 0x0F; // increment read index and wrap around 16 return size; } else { return 0; } }

I also wrote A 64bit memcpy in a separate ASM file

// rdi = source, rsi = destination, rdx = size memcpy64: push %rcx xchg %rdi,%rsi mov %rdx,%rcx shr $3,%rcx rep movsq mov %rdx,%rcx and $0x07,%rcx rep movsb pop %rcx ret

The interrupt handler:

unsigned short isr; INPORTW(isr,iobase+0x3E); OUTPORTW(0xFFFF,iobase + 0x3E); unsigned int status; unsigned char cmd=0; unsigned short size; unsigned short i; if (isr&1) // ROK { // It is very important to check this first because it's possible to get an interrupt // and still have cmd.BUFE set to 1. that caused me lots of problems like // reading bad status, causing buffer overflows INPORTB(cmd,iobase+0x37); while (!(cmd&1)) // check if CMD.BUFE == 1 { // if last frame overflowed buffer, this won't will start at rxBufferIndex%RX_BUFFER_SIZE instead of zero if (rxBufferIndex>=RX_BUFFER_SIZE) rxBufferIndex = (rxBufferIndex%RX_BUFFER_SIZE); status =*(unsigned int*)(rxbuf+rxBufferIndex); size = status>>16; memcpy64((char*)&rxbuf[rxBufferIndex],(char*)&rxBuffers[writeIndex][0],size); rxBufferIndex = ((rxBufferIndex+size+4+3)&0xFFFC); OUTPORTW(rxBufferIndex-16,iobase+0x38); writeIndex = (writeIndex+1)&0x0F; if (writeIndex==readIndex) { // Buffer overrun } INPORTB(cmd,iobase+0x37); } }

Sending

I found that Sending was easier than receiving. The first thing that needs to be done is to setup the buffer pointers in TSAD0-TSAD3. I'm not sure if these buffers require any special alignment but I've aligned mine on 2k boundaries.

Sending a frame

There are 4 TX buffers available. You should keep track of which one is free by incrementing an index everytime you send a frame. This way, you will know what buffer to use next time. You will need to copy your frame into the buffer pointed to by TSAD[CurrentSendIndex]. You will then need to write the size of the frame into TSD[CurrentSendIndex] and clear bit 13. Bit 13 is the OWN bit. It indicates to the card that this buffer is ready to be transmitted. Then you increment CurrentSendIndex to be ready for next time. At the next send, if TSD[CurrentSendIndex].bit13 is cleared, it means that the frame still belongs to the card and it wasn't transmitted. This would indicate a buffer overrun, your software is sending faster than what the card can handle.

unsigned long rtl8139_send(unsigned char* buf, unsigned short size) { if (size>1792) return 0; unsigned short tsd = 0x10 + (currentTXDescriptor*4); unsigned int tsdValue; INPORTL(tsdValue,iobase+tsd); if (tsdValue & 0x2000 == 0) { //the whole queue is pending packet sending return 0; } else { memcpy64((char*)&buf[0],(char*)&txbuf[currentTXDescriptor][0]);; tsdValue = size; OUTPORTL(tsdValue,iobase+tsd); currentTXDescriptor = (currentTXDescriptor+1)&0b11; // wrap around 4 return size; } }

Handling TX interrupt

Handling the interrupt is mostly done to detect send errors. I don't use it much. I won't go into details here, as the code explains pretty much everything.

unsigned short isr; INPORTW(isr,iobase+0x3E); OUTPORTW(0xFFFF,iobase + 0x3E); if (isr&0b100) //TOK { unsigned long tsdCount = 0; unsigned int tsdValue; while (tsdCount <4) { unsigned short tsd = 0x10 + (transmittedDescriptor*4); transmittedDescriptor = (transmittedDescriptor+1)&0b11; INPORTL(tsdValue,iobase+tsd); if (tsd&0x2000) // OWN is set, so it means that the data was transmitted to FIFO { if ((tsd&0x8000)==0) { //TOK is false, so the packet transmission was bad. Ignore that for now. We will drop it. } } else { // this frame is pending transmission, we will get another interrupt. break; } OUTPORTL(0x2000,iobase+tsd); // set lenght to zero to clear the other flags but leave OWN to 1 tsdCount++; } }

Documentation

These are good resources if you need more information on the rtl8139:

Get the full source code