Real World RF Range Testing

This paper outlines the procedure for a successful RF range test that provides quantitative data on how the RF link performs in its intended environment. This enables informed decisions about network architecture.

The paper also explains subtleties that might be overlooked during a RF range test that can affect performance. Although these principles apply to any radio, we give special attention to features of the 802.15.4 standard that can affect range test results.

The design of a RF range test is broken down, and specific elements are explained in a real-world range test using LSR’s RF modules.

So Many Variables

One of the most frequently asked questions about a given radio is “How far will this radio transmit?” Unfortunately, it is also one of the most difficult questions to answer, as so many variables apply. Besides obvious variables like transmit power and receive sensitivity, a host of other factors come into play. Without careful attention to understanding and controlling as many of these variables as possible, the results of an RF range test will leave more questions than answers.

For example, consider this common scenario. An engineer evaluating a radio puts the radio in a spot, walks some distance from it while monitoring an LED that indicates a received packet, and hopes to find a magical line on the ground at which the radio works on one side and doesn’t work on the other. In real life the link will become marginal at some point and continue to work intermittently, perhaps even improving at certain points, forcing the engineer to make a judgment call on how far the radio worked. A week later the same test performed by the same or a different individual yields drastically different results. What happened?

As we attempt to reconcile the results, questions come up. Even if the test was done outdoors, many factors apply. What antennas were used? How were they oriented? How far off the ground were they? How were the radios powered? If batteries were used, what was their voltage? Was the same version of firmware used? What was the over-the-air data rate? How many bytes were in the RF packet? What was the RF environment like during each test? How was it determined that the link was bad? Were RF acks and retries used? What obstructions were in the path of the radios? The answers to these questions reveal that the trials had little in common, and the results are inconclusive.

Add to this the complexities of an office or industrial environment. The RF environment is constantly changing as laptop computers move about and cordless phones or even microwave ovens are operated, affecting the 2.4GHz band in particular. Doors open and close, office equipment is turned on and off, equipment is moved, and so on. Also, the building’s physical construction certainly has an effect; an addition may make it appear that an outside wall of concrete, stucco or metal siding is an interior wall.

These situations illustrate why it is so hard to answer the question “How far will this radio transmit?” Clearly, as many variables as possible need to be controlled, and those that cannot be controlled at least need to be understood. When radio manufacturers list a range distance, be sure to ask:

How were those results obtained?

Is this the environment in which my radios will operate?

What was the performance of radios at this distance?

Good Range Test Design

Specifying Range

Most manufacturers specify range in an optimum situation, specifically line-of-sight outdoors. Besides putting their best foot forward, this enables them to repeatedly reproduce the test conditions and obtain similar results. This does not mean, however, that users of the radio can obtain the same results in a typical environment.

Before running the range test, determine the criteria for specifying how far the radio transmits. This is especially important when trying to compare different radios. Some may specify this as the farthest point at which 99% of packets are successfully transmitted. Others may plot out the average receive sensitivity over distance to determine range. A combination of both metrics could also be employed.

Write Everything Down

It should go without saying, but it’s easy to take shortcuts under the premise that we will remember. In reality, hours turn into days, and days into weeks, and we can’t recall the details. So take the time to write everything down. You could use the example at the end of this paper to create a form that ensures all important details are noted.

Collect Quantitative Data

A good range test will collect useful information, such as the number of packets transmitted, the number of packets received, RSSI, LQI, the amount of noise on the channel, and other test conditions such as timestamps on the packets, the number of bytes in the packet, and power supply conditions. Such information should be logged into a PC to enable analysis of the data collected.

Understand the Difference Between LQI and RSSI

RSSI is an acronym for Received Signal Strength Indicator. LQI is an acronym for Link Quality Indicator. Some use the terms interchangeably, but they are different. RSSI is an absolute value that can define the magnitude of the received signal, and it can be converted into units of dBm. LQI is a relative number between 0 and 255 that represents the quality of the received signal. Since there are no official rules on how it is calculated, every stack vendor usually calculates LQI differently. It is generally a function of RSSI and possibly other indicators such as the quality of the modulation on the received signal.

When range testing a radio, it is generally good to track both RSSI and LQI and to understand how the LQI is being calculated. A strong RSSI but weak LQI (assuming LQI is factoring in the quality of the modulation) would indicate the presence of RF noise or interference.

Understand Acknowledgment Requests and Retries

Many radio standards today, such as 802.25.4, employ a RF acknowledgment/retry mechanism. When they are mentioned, it is important to understand that they can exist in the MAC layer and the application layer of software. In 802.15.4, acks/retries can be enabled on a packet-by-packet basis in the MAC. If a message is sent with MAC layer ack, the recipient of the message will send a very small RF acknowledgment back to the sender, confirming receipt. If the sender does not receive this ack, it will resend (retry) the message again, up to three times. If you are doing the math, the means that the same message could be transmitted up to four times: the original message and up to three retries.

Figure 1: MAC layer acks/retries

It is often assumed that if a message is retried, it was not heard by the recipient. In practice, it is not uncommon for the acknowledgment to go unheard by the originator, prompting the originator to resend the message. This means that it’s possible for the recipient to hear the same message multiple times.

Figure 2

It is important to consider that using MAC layer acks/retries may create a false sense of security regarding RF-link quality. For example, if the ack/retry mechanism results in each message being sent two or three times, it may appear that the link is very good when in reality it is on the fringes. A clue that this is happening could be gathered from statics [statistics?] kept on each side of the link. If the originator side shows 100 packets were sent out and the recipient shows 300 messages were received, this is a sure sign that the network relied heavily on the ack/retry mechanism.

Although not the focus of this paper, application layer acks/retries have more practical value in hopping networks or in networks where extremely high reliability is required. The MAC layer acks/retries only guarantee that the message made it to the next hop – not the final recipient. Also, even in single-hop systems, this method does not guarantee that the message made it to the application. For example, a node could receive an RF message, send the MAC layer ack, and then try to allocate memory to send the message to a PC or display but run out of memory. The message originator would have no knowledge of this problem.

Figure 3: Pitfall of MAC-only acks

LSR RF Range Test

LSR has developed a protocol for evaluating range of its radio modules, and the principles contained therein can be used to conduct a successful range test. One part of the process involves using a simple form, like the one at the end of this paper, to write down important details about the test, including details that are easy to overlook. The second part of the process combines the firmware on the radio module being tested with PC software to display and record the results. The range test is also designed to achieve the following goals:

Obtain quantitative range test results (Packet Error Rate, RSSI, LQI, Background RF Noise).

Establish the ability to restart data collection remotely – that is, look at performance in various locations without having to walk back to a PC to restart the test.

Enable a single user to perform the entire test.

Provide a means to log the results.

Record as many variables (such as battery voltage) as possible.

Use only one PC.

Materials for RF Range Test

The range test in this paper was performed with two of the ModFLEX series radios, the ProFLEX01 (2.4 GHz DSSS) and SiFLEX02 (900 MHz DSSS). Development kits for the radios are commercially available. The module firmware and PC software (ModFLEX Test Tool Suite) are available online.

Over-the-Air Packet Structure

Necessary parts of range test results are transmitted over the air to allow either end of the system to collect data. Each time the master transmits a packet, it increments its packet ID. If the “application ack” is set to 1, the slave device will transmit a packet back to the master, making it a round-trip test. The following example shows what these packets would look like.

Master-to-Slave Packet

The packet ID indicates how many packets have been transmitted by the master.

The power supply voltage of the master is sent to the slave.

If the slave receives the packet, these can be determined: 1 packet was transmitted, 1 packet was received (100% success); and RSSI, LQI, and battery voltages.

Slave information is 0 as this information needs to be filled in from the slave.

Table 1: First master-to-slave packet

Slave-to-Master Packet

The slave appends information into the packet: RSSI and LQI of the master-to-slave packet; battery voltage.

If the master receives the packet, these can be determined: 1 packet was transmitted, 1 packet was received (100% success); RSSI, LQI, and battery voltages of itself and the slave.

 

Table 2: First slave-to-master packet

Missed Packets

If the slave misses several packets, the statistics will correct themselves as soon as another packet is received. For example, if the slave does not hear packets 90-99 but does hear packet 100, the over-the-air message would look like the one in Table 3. It would be determined that the master sent 100 messages but the slave heard only 90.

Table 3

Restarting Test in the Field

Smarts are built into the range test application to start/restart the test if the packet ID of the received message is less than the packet ID of the previous message. This would allow a user to restart the packet-error-rate calculation at distance intervals without having to reset both ends of the test. Table 4 provides an example of how this would look in the over-the-air packet.

Table 4

RF Acknowledgments and Retries

In the LSR range test, application layer acknowledgments can be turned on or off. Turning them off makes the test one-dimensional; turning them on creates a round-trip test. Application layer acks/retries can be used in conjunction with MAC layer acks/retries.

Table 5

Table 6

Figure 5: Application acks disabled

Host Serial Message Packet Structure

Applicable information from the received range test RF packet and additional information is formatted into a serial message and passed on to the PC. The ModFLEX Test Tool Suite uses this information to display and log the results of the range test. More information on the serial message is available in the respective modules’ Host Protocol Guide at the LSR website: www.lsr.com.

Figure 6: Range test serial message

Figure 7: Range test results in the ModFLEX Test Tool Suite

Sample Range Test Form for SiFLEX02 Testing

Date of test	June 1, 2010
Name of person performing test	Christopher Hofmeister, LSR
Location	Star Lake, Vilas County, Wisconsin. See Figure 10: Map of test area.
Radio tested	SiFLEX02 (900 MHz DSSS)
Antenna(s) tested	Wire antenna, +2dBi dipole (LSR P/N 001-0002)
Data rate	40kbs
Firmware version	2.0 5-29-2010
Hardware version	2.0
Channel tested	5
Output power	+24dBm
ED results	0 – See Figure 14: SiFLEX02 energy scan results
MAC level retries	Off
Application level retries	On – making this a round-trip test in which the results of each side are recorded.
Power sources	The slave device was powered by two AA batteries. The batteries were verified to be 2.9V or greater at the beginning of the test. The master device was powered by the USB of a laptop computer.
Antenna orientation	Both the wire and dipole antennas were kept in the vertical position.
Height of devices off ground	The slave device was placed on an outbuilding on the edge of the lake, 10' above water level. See Figure 12: Slave device location. The master device was 9' above water level. The tester stood with his arm extended vertically and held the master device in his hand, always pointing toward the slave device.
Description of test	The master device was carried in a boat and connected to a laptop, which allowed viewing and recording the results. Increments of 0.25 mile were marked on a handheld GPS device the length of the lake (1.5 miles). Every 0.25 miles, the boat was stopped, and 500 packets were exchanged between the devices. The goal was to plot RSSI vs. distance in an open-air, line-of-sight environment. See Figure 11: View across lake (1.5 miles) looking toward slave device.

SiFLEX02 Results

Results were graphed for RSSI vs. distance on each antenna type and data-rate. The results from each end of the test were graphed: what the slave saw from the master (MàS) and what the master saw from the slave (SàM).

At 40kbs, the SiFLEX02 with the +2dBi dipole antenna had adequate link margin to the end of the lake at 1.5 miles. If the data were extrapolated, it is reasonable to assume the link margin would be adequate to at least 2.0 miles.

At 40kbs, the SiFLEX02 with the wire antenna had adequate link margin to the end of the lake at 1.5 miles. If the data were extrapolated, it is reasonable to assume the link margin would be adequate to at least 1.75 miles.

Figure 9: RSSI vs. distance for F-antenna and +2dBi dipole antenna

Appendix: Supporting Documentation

Figure 10: Map of test area

Figure 12: Slave device location

Figure 14: SiFLEX02 energy scan results

Christopher Hofmeister, a senior software engineer at LSR, has spent more than 15 years in the low-power wireless industry. He has been involved in hardware and firmware design as well as testing of low-power RF transmitters, receivers and transceivers for both engineering validation and production testing.

Contact LSR for RF testing services.