Rethinking the Photometric Data File Format

by | 2019 Feb Fri | General | 0 comments

Ian Ashdown, P. Eng., FIES, Senior Scientist, SunTracker Technologies Ltd.

Published: 2017/03/01 

If you perform lighting design calculations today, you can thank the efforts of the IES Computer Committee (IESCC) some thirty years ago. Its members recognized an industry need, and so developed and published IES LM-63-86, IES Recommended Standard File Format for Electronic Transfer of Photometric Data. With the growing popularity of the IBM Personal Computer for business applications, it was an idea whose time had come.

The need was clear: Lighting Technologies (Boulder, CO) had released its Lumen Micro lighting design and analysis software product in 1982, and luminaire manufacturers needed to provide photometric data for their products. For them, IES LM-63 was a god-send in that it established an industry-standard file format7.

In keeping with the technology of the time, the file format was human-readable ASCII text, something that could be printed with a dot-matrix printer. It also resulted in files of only a few kilobytes, a definite advantage when data files were transferred by mail on 5-1/4 inch floppy diskettes capable of holding 360 kilobytes of data. The file format itself revealed something of its origins by limiting line lengths to 80 characters — the width of an IBM Hollerith punch card in the 1960s (FIG. 1).

FIG. 1 – IBM Hollerith 80-character punch cards

Thirty years later, our personal computers are one thousand times faster, with one million times the memory capacity and ten million times more data storage capacity. Data is transferred by fiber optic cable and satellite links at gigahertz rates … and we are still using IES LM-63 photometric data files!

The “we,” of course refers mostly to North America. In Europe, the equivalent file format is EULUMDAT, which was introduced in 1990 for use with Microsoft’s MS-DOS 3.0 operating system14. Again, in keeping with the technology of the time, it was also human-readable ASCII text.

It is a testament to something … exactly what is unclear … that these two file formats have met the lighting industry’s needs for so long. Coming from an era of floppy diskettes and dial-up modems with acoustic couplers (FIG. 2), they should have become extinct decades ago. (The Chartered Institute of Building Services Engineers in the United Kingdom introduced its CIBSE TM14 file format specification in 1988, but it has since slipped into obscurity2.)

FIG. 2 – Modern communications technology circa 1986.

To be fair, LM-63 was revised in 1991, 1995, and 2002. These revisions, however, at best tweaked the file format specification to resolve various ambiguities and add a few minor features. What we have today is basically what was published in 1986, a time when the pinnacle of lamp technology was the compact fluorescent lamp with an electronic ballast.

If the LM-63 file format has an advantage, it is that it is an ANSI/IES standard that is maintained by an internationally recognized standards organization. EULUMDAT, on the other hand, is a de facto standard that has been essentially frozen in time since its publication in 1990[á]. Without the authority of a standards organization such as ANSI/IES or CEN (European Committee for Standardization) to maintain the file format, it can never be revised.

The problem is that while LM-63 and EULUMDAT are still useful in terms of characterizing architectural and roadway luminaires, the lighting industry has moved beyond luminous intensity distributions. As professional lighting designers, we now have to consider color-changing luminaires, theatrical lighting, human-centric lighting, horticultural lighting, ultraviolet sterilization units, radiant heating devices, and more. We need to consider spectral power distributions, radiant intensity, photon intensity, S/P ratios, melanopic lumens, color rendition metrics … the list goes on and on.

The LM-63 and EULUMDAT file formats are clearly incapable of characterizing light sources and luminaires for these applications. It is therefore time, indeed well past time, to rethink the photometric data file format.

Standards Development

In September 2016, the IESCC initiated a project to develop a new photometric data format from first principles. As is often the case with such projects, one or two members write an initial draft based on their expertise and knowledge. This draft document is reviewed, edited numerous times, and voted upon by the committee members. If approved as a project, the proposed project is again reviewed and voted upon by the IES Board of Directors. In January 2017, technical committee project IES TM-xx, Standard Format for the Electronic Transfer of Luminaire Optical Data, was officially approved (FIG. 3).

FIG. 3 – IESCC project summary

Luminaire Component Data

Some readers may recognize this proposed standard from a previous incarnation known as IESNA LM-74-05, Standard File Format for the Electronic Transfer of Luminaire Component Data8. The IESCC worked on the development of this document for nearly a decade prior to its publication in 2005. It was ambitious effort to combine all aspects of luminaires into a single file, including far-field photometry, lamp and ballast information, physical geometry, construction materials and finishes, CAD drawings and photographs, and more.

Unfortunately, it was too ambitious. Despite the first release being focused on lamp data, the standard was never adopted by its intended audience of luminaire manufacturers, architects and engineers, lighting product specifiers, photometric testing laboratories, and lighting software developers. To the frustration of the IESCC members, the lighting industry at the time did not see a need for such a standard.

Today, we might look upon LM-74-05 as being an early example of a specialized building information management (BIM) schema, one that focused on a small subset of typically much larger datasets. (A document schema is conceptually equivalent to a file format.) The Green Building XML Schema (gbXML) for BIM applications provides an excellent example. Quoting from the gbXML Web site:

“The Green Building XML schema, or ‘gbXML,’ was developed to facilitate the transfer of building information stored in CAD-based building information models, enabling interoperability between disparate building design and engineering analysis software tools. This is all in the name of helping architects, engineers, and energy modelers to design more energy efficient buildings.”

Unfortunately for lighting design professionals, the gbXML schema has an XML “element” (see below) called “Photometry,” whose description reads:

“This element has been left open for use with other photometry definitions. Photometric data is required for various forms of lighting analysis. This tag provides a way for the photometric data to be passed. Since this can be done in a variety of ways (iesna LM-63, cibse TM14, ELUMDAT, etc.) a specific format is not being specified.”

Defining a new luminaire optical data format that is compatible with the gbXML schema therefore serves a clear and present need.

Understanding XML

The advantage of gbXML is that it is based on the international data exchange standard XML (eXtensible Markup Language)15. The details of this standard are complex and exhausting, but basically every XML document consists of text strings called “elements” such as:

where the data is surrounded by begin and end “tags.”

These elements can be arranged in a hierarchy, such as:

In this example, the <person> element is the “parent,” and any elements within it are its “children.”

Building on this simplest of representations, virtually any type of data can be unambiguously represented within an XML document. If a person or computer program reading an XML document encounters an unknown element tag, the element and its children (if any) can simply be ignored.

This, of course, is the problem with including LM-63 or EULUMDAT text files verbatim (i.e., as a multiline text string) within gbXML or similar BIM documents. Yes, it can be done, but the computer program reading the document needs to be able to somehow identify and read these files. Designing IES TM-xx as an XML document resolves this problem.

Having chosen a suitable representation for TM-xx, we can now consider what it needs to represent.

Luminaire Optical Data

IES TM-xx represents the luminaire optical data in four sections:

  1. Header
  2. Luminaire
  3. Equipment
  4. Light source

Header

The header section includes information that is currently available in LM-63 and EULUMDAT files:

  • Manufacturer
  • Catalog number
  • Description
  • Test laboratory
  • Report number
  • Report date
  • Document creator
  • Document creation date
  • Unique identifier
  • Comments

Most of these elements are self-explanatory, with the exception of the “unique identifier” element. One of the problems with current photometric data files is that there is no version control. If a company reissues photometric data for a product, there is no way of distinguishing between files other than their file creation dates. If the files are copied for any reason, these dates can change.

The unique identifier element is a “Universally Unique IDentifier” (UUID) that uniquely identifies the TM-xx document, regardless of whether it has been copied as a file. While it does not prevent someone from intentionally modifying the document data, it at least solves the problem of multiple files with the same name.

The IESCC is currently considering the addition of search terms and possibly CAD symbols to the header section. These and other details may therefore result in changes to the draft release of TM-xx, but the basic structure discussed here will remain.

Luminaire

TM-xx represents the luminaire as a rectangular box or cylinder. The luminaire section therefore lists the dimensions of these geometric objects as length, width, and height. In addition, each face may include an emission area. These areas are useful for calculating visual glare metrics such as the CIE Unified Glare Rating (UGR)5, and also for modeling the luminaire as one or more area sources or arrays of point sources for lighting calculations and visualization.

The luminaire section also includes the light center position with respect to the geometric center of the luminaire. (The light center represents the fixed position about which the goniometer rotates while performing intensity distribution measurements.)

Equipment

The equipment section describes the laboratory equipment used to perform the luminaire optical data measurements. These can include:

  • Goniometer (intensity measurements)
  • Integrating sphere (flux measurements)
  • Spectroradiometer (spectral power distribution measurements)

and detailed information specific to these instruments.

Light Source

Photometric data files assume that the luminaire includes one or more removable lamps, but this concept does not apply to solid state lighting, which may have removable LED modules or non-removable LED arrays. For the purposes of TM-xx, these are collectively referred to as “light sources.” Following LM-63 and EULUMDAT, the information pertaining to them may include (as applicable):

  • Quantity
  • Description
  • Catalog number
  • Rated lumens
  • Input wattage
  • Tilt angle

In addition, the information may include correlated color temperature (CCT) values, color rendering metric values (Ra and R9 for CIE Colour Rendering3 and Rf and Rg for IES Color Rendition11), and scotopic-to-photopic lumens (S/P) ratios9. (Note that these values may need to be expressed as ranges for variable color temperature light sources.)

There are actually 14 CIE Colour Rendering Special Indices (R1 to R14), which may be required for special purposes. These can either be calculated by the user from the measured spectroradiometric data for the light source (see below), or represented by custom XML elements.

Spectroradiometric Data

A key requirement of the light source section is to represent the spectral power distribution (SPD) of the light source. Following IES TM-27-14, IES Standard Format for the Electronic Transfer of Spectral Data10, the measured spectral radiant flux is reported for each wavelength.

Most SPDs are reported with constant wavelength intervals (e.g., 5 nm), but TM-xx does not impose such a restriction. Consequently, both continuous and line emission spectral features can be represented with arbitrary wavelength precision.

Intensity Data

With photometric data files, most of the data represents the luminous intensity measurements for vertical and horizontal angles. The same is true for TM-xx documents in the light source section except that, depending on the application, the intensity measurements may be based on luminous flux, radiant flux, photon flux[ß], or spectral radiant flux.

Luminous intensity distributions are expressed in lumens per steradian (i.e., candela), and are most useful for architectural and roadway lighting applications.

Radiant intensity distributions are expressed in watts per steradian, and are most useful in characterizing ultraviolet and infrared radiation sources for applications such as UV sterilization and radiant heating.

Photon intensity distributions are expressed in micromoles per steradian per second, and are most useful for horticultural lighting applications1.

Both radiant and photon intensity are measured over a specified range of wavelengths. When photon intensity is measured over the range of 400 nm to 700 nm, it is equivalent to photosynthetically active radiation (PAR)1.

Spectral radiant intensity distributions assign an SPD to each measurement for vertical and horizontal angles. Expressed in watts per steradian per nanometer, they are useful for representing the variation in color over viewing angle, such as occurs with phosphor-coated white light LEDs.

Finally, each intensity measurement is expressed as (for example):

By explicitly expressing the vertical and horizontal angles for each measurement, there is no requirement for the data to be organized as a two-dimensional array of vertical angles and horizontal planes. This is important because some robotic goniometers are capable of measuring angular positions on a geodesic sphere and other complex angular patterns.

Exclusions

IES TM-xx differs from its predecessor LM-74-05 in that it focuses exclusively on luminaire optical data. This necessarily excludes other luminaire components and characteristics, including:

  • Detailed physical dimensions
  • Mechanical and structural data
  • Materials and finishes
  • Building code certifications
  • CAD drawings
  • Photographs and renderings
  • Electronic ballasts and drivers
  • Lighting controls and sensors

It would certainly be possible to include this information, but it comes at a price. Every time a component option is added to a product, it increases the number of product variations exponentially. If, for example, a lighting control has four ordering options, this potentially results in 16 different TM-xx documents.

With this, the design philosophy for TM-xx follows what Albert Einstein purportedly once said: “Everything should be made as simple as possible, but no simpler.” Given the purposeful extensibility of XML, it is always possible to add elements with custom tags for specific purposes. To avoid conflicts with identical tag names being used by other companies, an XML namespace can be used to uniquely identify the custom tags. With this, TM-xx is being designed to be “as simple as possible but no simpler.”

This design philosophy also extends to the intensity data. TM-xx optionally reports luminous, radiant, photon, and spectral radiant intensity, but not, for example, melanopic intensity that is useful in human centric lighting applications13. The reason is that if you report melanopic intensity, you should arguably also report cyanopic, chloropic, erythropic, and rhodopic intensity to represent to the responsivity of short-wavelength, (blue), medium-wavelength (green), and long-wavelength (red) cones and rods respectively in the human retina. (Melanopic intensity represents the responsivity of intrinsic retinal ganglion cells, or ipRGCs, to retinal irradiance.) This is, of course, an extreme example, but it illustrates the complexities that can arise in trying to satisfy every requirement.

Luminous, radiant, photon, and spectral radiant intensity are optionally reported because they are the most commonly used metrics in architectural, roadway, and horticultural lighting applications. All other intensity metrics can be calculated, if necessary, by appropriately weighting the spectral radiant intensity data.

Files versus Documents

Thirty years ago, almost all data was stored on magnetic media — floppy disks, hard disks, and magnetic tape. Data, whatever its form, was organized in the form of files. It therefore made sense to refer to photometric data files and file formats.

Today, data is stored on a variety of media, including magnetic, optical, solid state, and holographic devices. Long-term storage of data still requires data files and file formats, such as the default NTFS file system used by Microsoft Windows operating systems. However, the data itself has become somewhat more amorphous. How it is organized better described as a document, a symbolic representation of the data.

Using gbXML as an example, an architect or engineer may assemble a temporary BIM document by linking together information from various manufacturers. The BIM program sends requests to the manufacturers’ servers, which may in turn assemble BIM documents to be returned as XML documents. The documents are compressed for transmission, so that the document format is converted into a more compact representation. More to the point, the document may never exist as a physical file.

With this, TM-xx defines a standard format for XML documents. The term “file” is properly relegated to the era of floppy disks and acoustic modems.

Why Not JSON?

Computer-savvy readers may well ask, “Why XML and not JSON?” After all, JSON is an alternative computer markup language that is widely used to exchange data between browsers and servers12. Compared to XML, it is a much simpler and less verbose language that typically results in smaller documents. It also natively supports two-dimensional data (i.e., matrices) such as luminous intensity distributions, which are more difficult to represent in XML.

The answer is that the electronic exchange of data between computer systems typically involves compressed documents, often with the ZIP file format. In the compression process, the element tags are represented by single symbols, which typically results in document compression ratios of 10:1. More to the point, compressed XML and JSON documents representing the same data are typically the same size. With this, the ability to embed TM-xx documents in XML-based BIM documents outweighs any advantages of JSON.

International Standards

Referring once again to FIG. 3, note the phrase “international standard.” There have been several attempts in the past to develop an international standard for photometric data formats, including CIE 102-1993, Recommended File Format for the Electronic Transfer of Luminaire Photometric Data4 and EN 13032-1:2004+A1:2012, Light and Lighting — Measurement and Presentation of Photometric Data of Lamps and Luminaires — Part 16. Despite being an explicitly international file format developed by representatives of eleven countries, CIE 102 was never adopted for commercial use. Sadly, the same fate seems to have befallen EN 13032-1.

This is, unfortunately, the fate of many standards. Companies and individuals volunteer their time and expertise to develop standards that meet perceived industry needs, but the industry in question is the final arbiter of what its needs are. If existing standards are sufficient, it is often difficult to convince manufacturers to abandon them in favor of a new and untried standard. Good examples of this are CIE 102-1993, EN 13032-1, and IES LM-74-05, but there are many others.

Recognizing this problem, the IES Computer Committee has chosen to work directly with its international colleagues, including lighting software companies, luminaire manufacturers, testing laboratories, lighting professionals, and academia with expertise in both architectural and horticultural lighting. It is further using social media to communicate its activities and invite feedback from several thousand lighting professionals. More than any other standard, TM-xx is being designed by those who will most benefit from its adoption and use.

Finally — and this is perhaps a key point — IES TM-xx has been explicitly designed to be forward compatible with IES LM-63, EULUMDAT, and CIBSE TM14. That is, it will be possible to automatically batch convert previous photometric data files into TM-xx documents with insignificant loss of information. Lighting software companies, luminaire manufacturers, and testing laboratories will therefore be encouraged but not required to transition their workflow and photometric data to TM-xx. In the meantime, lighting design software will be able to seamlessly support both photometric data files and TM-xx documents.

Summary

The intent of this article has been to: 1) review the history of photometric data file formats; 2) describe the ongoing efforts of the IES Computer Committee and its partners to develop an international standard for architectural, roadway, and horticultural lighting; and 3) describe both the design philosophy and the international effort behind the development of IES Technical Memorandum TM-xx, Standard Format of the Electronic Transfer of Luminaire Optical Data.

Simply put, the lighting industry currently relies on photometric data file formats that were developed three decades ago. IES TM-xx is being designed for today and the future.

References

  1. ASABE. 2017. Definition of Metrics of Radiation for Plant Growth (Controlled Environment Horticulture) Applications. St. Joseph, MI: American Society of Agricultural and Biological Engineers. (In press.)
  2. CIBSE. 1988. TM14:1988, CIBSE Standard File Format for the Electronic Transfer of Luminaire Photometric Data. London, UK: Chartered Institution of Building Services Engineers.
  3. CIE 13.3-1995. Method of Measuring and Specifying Colour Rendering Properties of Light Sources. Commission Internationale de l’Eclairage, Vienna, Austria.
  4. CIE 102-1993. Recommended File Format for Electronic Transfer of Luminaire Photometric Data. Commission Internationale de l’Eclairage, Vienna, Austria. (Originally published in CIE Journal 6(1):23-31, 1987.)
  5. CIE 117-1995. Discomfort Glare in Interior Lighting. Commission Internationale de l’Eclairage, Vienna, Austria.
  6. EN. 2012. European Standard EN 13032-1:2004+A1:2012, Light and Lighting — Measurement and Presentation of Photometric Data of Lamps and Luminaires — Part 1: Measurement and File Format. Brussels, Belgium: European Committee for Standardization.
  7. IES LM-63-02. Standard File Format for Electronic Transfer of Photometric Data. New York, NY: Illuminating Engineering Society of North America.
  8. IES LM-74-05. IESNA Standard File Format for the Electronic Transfer of Luminaire Component Data. New York: Illuminating Engineering Society of North America.
  9. IES TM-12-12. Spectral Effects of Lighting on Visual Performance at Mesopic Light Levels. New York, NY: Illuminating Engineering Society of North America.
  10. IES TM-27-14. IES Standard Format for the Electronic Transfer of Spectral Data. New York, NY: Illuminating Engineering Society of North America.
  11. IES TM-30-15. IES Method for Evaluating Light Source Color Rendition. New York, NY: Illuminating Engineering Society of North America.
  12. IETF. 2014. The JavaScript Object Notation (JSON) Data Interchange Format. Request for Comments 7159, Internet Engineering Task Force. (Available from https://tools.ietf.org/html/rfc7159.)
  13. Lucas, R. J., S. N. Peirson, D. N. Berson, T. M. Brown, H. M. Cooper, C. A. Czeisler, M. G. Figueior, P. D. Gamlin, S. W. Lockley, J. B. O’Hagan, L. L. A. Price, I. Provencio, D. J. Skene, and G. C. Brainard. 2013. “Measuring and Using Light in the Melanopsin Age,” Trends in Neuroscience 37(1):1-9.
  14. Stockmar, A. W. 1990. “EULUMDAT — ein Leuchtendatenformat für den europäischen Beleuchtungplaner,î Tagungsband Licht í90, pp. 641-644.
  15. W3C Extensible Markup Language (XML) 1.0, Fifth Edition (http://www.w3.org/TR/REC-xml).

[á] The EULUMDAT file format specification is available from http://www.helios32.com/Eulumdat.htm.

[ß] Photon flux, also commonly referred to as quantum flux, is the rate of flow of photons. Radiant flux, by comparison, is the rate of flow of energy. The energy of a photon is inversely proportional to its wavelength, so quantum flux is not directly comparable to radiant flux.