GNU IDN Library - Libidn
Introduction
GNU Libidn is a fully documented implementation of the Stringprep, Punycode and IDNA 2003 specifications. Libidn's purpose is to encode and decode internationalized domain names. The native C, C# and Java libraries are available under the GNU Lesser General Public License version 2.1 or later.
Please be aware that GNU libidn2 is the successor of GNU libidn. It comes with IDNA 2008 and TR46 implementation and also provides a compatibility layer for GNU libidn.
The library contains a generic Stringprep implementation. Profiles for Nameprep, iSCSI, SASL, XMPP and Kerberos V5 are included. Punycode and ASCII Compatible Encoding (ACE) via IDNA are supported. A mechanism to define Top-Level Domain (TLD) specific validation tables, and to compare strings against those tables, is included. Default tables for some TLDs are also included.
The Stringprep API consists of two main functions, one for converting data from the system's native representation into UTF-8, and one function to perform the Stringprep processing. Adding a new Stringprep profile for your application within the API is straightforward. The Punycode API consists of one encoding function and one decoding function. The IDNA API consists of the ToASCII and ToUnicode functions, as well as an high-level interface for converting entire domain names to and from the ACE encoded form. The TLD API consists of one set of functions to extract the TLD name from a domain string, one set of functions to locate the proper TLD table to use based on the TLD name, and core functions to validate a string against a TLD table, and some utility wrappers to perform all the steps in one call.
The library is used by, e.g., GNU SASL and Shishi to process user names and passwords. Libidn can be built into GNU Libc to enable a new system-wide getaddrinfo flag for IDN processing.
Libidn is developed for the GNU/Linux system, but runs on over 20 Unix platforms (including Solaris, IRIX, AIX, and Tru64) and Windows. The library is written in C and (parts of) the API is also accessible from C++, Emacs Lisp, Python and Java. A native Java and C# port is included.
Also included is a command line tool, several self tests, code examples, and more, all licensed under the GNU General Public License version 3.0 or later.
Table of Contents
- Introduction
- News
- Try it
- Documentation
- Downloading
- Support
- Development
- Bugs
- Related implementations
- How to use it?
- Libidn2
News
Note that new releases are only mentioned here if they introduce a major feature or is significant in some other way. Read the help-libidn mailing list if you seek more frequent announcements.
- 2012-01-10: An infloop bug was fixed for the pr29 functions. The library has been relicensed to dual-GPLv2+|LGPLv3+. See the Libidn 1.24 announcement.
- 2011-05-04: Quality Assurance improvements: we publish clang-analyzer reports for the library.
- 2011-04-20: An IDNA2008 implementation is announced called libidn2.
- 2008-10-07: Quality Assurance improvements: we publish cyclomatic code Complexity charts and self-test code coverage charts.
- 2007-07-31: Version 1.0 is released, to indicate that Libidn is now considered stable. It has been used in production for several years with only minor issues found.
- 2007-05-31: Libidn is now developed in git instead of cvs, there is a public savannah git repository.
- 2006-06-07: Translation of error messages are working, and the library has been ported to Windows using MinGW.
- 2005-12-03: Version 0.6.0 include a native C# port, contributed by Alexander Gnauck.
- 2004-11-08: GNU/Linux distribution Fedora Core 3 includes Libidn version 0.5.6.
- 2004-10-02: Version 0.5.6 include functions
(e.g.,
idna_strerror
) to translate from return codes to human readable text strings. - 2004-06-26: Version 0.5.0 include a module to detect "problem sequences" for normalization as discussed in PR-29.
- 2004-06-01: Version 0.4.8 include a native Java port, thanks to Oliver Hitz.
- 2004-04-30: People interested in the specifications behind libidn may be interested in a proposed change to NFKC by the Unicode Consortium. I have posted a message to the IDN WG mailing list asking for opinions on this, but apparently the list moderator is ignoring it.
- 2004-03-27: Recently a
patch
to GNU Libc has
been incorporated, extending the
getaddrinfo
API based on my writeup. The API is being standardized. - 2004-02-28: A NetBSD package exists.
- 2004-02-28: Version 0.4.0 includes an experimental API for (parts of) the TLD functionality described in draft-hoffman-idn-reg.
- 2004-01-30: A Perl module Net::LibIDN that provide Perl bindings for Libidn is available, thanks to Thomas Jacob. The page also include a patch that add TLD specific awareness to Libidn.
- 2004-01-06: A FreeBSD ports package is available, thanks to Kirill Ponomarew.
- 2004-01-01: Savannah had problems last month, and still isn't operating fully. CVS has been moved to a private machine, a read-only mirror of it will hopefully be available via Savannah in the future.
- 2003-10-29: A project with the goal of providing PHP bindings of the Libidn API has been started by Turbo Fredriksson.
- 2003-10-11: Precompiled binaries for Mandrake 9.2 available built as part of glibc, and as a RPM package, thanks to Oden Eriksson.
- 2003-10-02: Version 0.3.1 fixes all problems discovered during IDNConnect.
- 2003-06-26: Precompiled binaries for Cygwin available from https://anfaenger.de/cygwin/libidn/, thanks to Gerrit P. Haase.
- 2003-02-26: Version 0.1.11 includes a command line tool and a Emacs Lisp interface.
- 2003-02-21: Debian includes libidn, thanks to Ryan M. Golbeck.
- 2003-02-12: Version 0.1.7 uses official IDNA ACE prefix 'xn--'.
- 2003-01-28: Version 0.1.5 can be built as an add-on to GNU Libc, available are detailed instructions and example code demonstrating the new getaddrinfo() API.
- 2003-01-08: Added a simple
patch demonstrating support for IDN in the
GNU
InetUtils
ping
utility. - 2003-01-05: Version 0.1.0 released with Punycode and IDNA.
- 2003-01-03: Libidn is an official GNU project.
- 2002-12-26: Moved project to savannah. Initiated renaming of library from "libstringprep" to "libidn" as the next release will implement Punycode and IDNA too.
- 2002-12-13: Version 0.0.8 is ported to 20+ platforms, including Microsoft Windows.
- 2002-11-07: Version 0.0.2 is now used by GNU SASL.
- 2002-11-05: Initial release of version 0.0.0.
Information on what is new in the library itself is found in the NEWS file (live version).
Try it
A web interface to libidn is available online. Try libidn before you buy it.
A simple IDN web server is also available.
Documentation
Refer to the Libidn Manual web page for links to the manual in all formats; however, quick links to the most popular formats:
You may also be interested in a preliminary document with Nameprep and IDNA test vectors.
See also the various standard texts:
- IDNA specification
- Punycode specification
-
Stringprep specification
- Standard profiles
- Expired profiles
- TLD specification
- IANA Registry for Stringprep Profiles
Downloading
Libidn can be found on https://ftp.gnu.org/gnu/libidn/ [via HTTPS] and ftp://ftp.gnu.org/gnu/libidn/ [via FTP]. It can also be found on one of our FTP mirrors; please use a mirror if possible.
All official releases are signed with an OpenPGP key with fingerprint 0xB565716F or with OpenPGP key 0x08302DB6A2670428.
Support
A mailing list where users of Libidn may help each other exists, and you can reach it by sending e-mail to help-libidn@gnu.org. Archives of the mailing list discussions, and an interface to manage subscriptions, is available through the World Wide Web at https://lists.gnu.org/mailman/listinfo/help-libidn.
If you are interested in paid support for Libidn, or sponsor the development, please contact me. If you provide paid services for Libidn, and would like to be mentioned here, also contact me.
If you find Libidn useful, please consider making a donation. No amount is too small!
Development
There is a Savannah Libidn project page. You can check out the sources by using git as follows:
$ git clone git://git.savannah.gnu.org/libidn.git
The online git interface is available.
For every release, we publish cyclomatic code complexity charts for the package. There is also self-test code coverage charts available. Finally, clang-analyzer output is also available.
Bugs
Report all problems to bug-libidn@gnu.org, but please read the manual on how to report bugs first.
Related implementations
The following is a list of links to other free IDN, or otherwise related, implementations. The list is not conclusive, suggestions appreciated.
Projects using GNU Libidn include:
- GNU Emacs, in the Gnus news reader.
- GNU Libc
- GNU Shishi
- GNU SASL
- jabberd
- Mutt mail reader.
- Elinks web browser
- Gloox, a Jabber/XMPP library
- KDE, for all domain name lookups
- Net::LibIDN, perl bindings
- LibIDN Ruby bindings
- cURL
- PHP IDNA Extension
Projects using libidn2 include:
Let us know about more projects that use GNU Libidn!
How to use it?
Read data from user, convert it to UTF-8 and then pass it to stringprep(). Example code below (it is included in the distribution as example.c). To simplify compiling, use libtool and pkg-config. More information and more examples are included in the manual.
See also the other example*.c files in the source distribution on how to use other features of the library (punycode, IDNA).
#include <stdio.h> #include <stdlib.h> #include <string.h> /* * Compiling using libtool and pkg-config is recommended: * * $ libtool cc -o example example.c `pkg-config --cflags --libs libidn` * $ ./example * Input string encoded as `ISO-8859-1': ª * Before locale2utf8 (length 2): aa 0a * Before stringprep (length 3): c2 aa 0a * After stringprep (length 2): 61 0a * $ * */ int main(int argc, char *argv[]) { char buf[BUFSIZ]; char *p; int rc, i; printf("Input string encoded as `%s': ", stringprep_locale_charset ()); fflush(stdout); fgets(buf, BUFSIZ, stdin); printf("Before locale2utf8 (length %d): ", strlen(buf)); for (i=0; i < strlen(buf); i++) printf("%02x ", buf[i] & 0xFF); printf("\n"); p = stringprep_locale_to_utf8 (buf); if (p) { strcpy(buf, p); free(p); } else printf("Could not convert string to UTF-8, continuing anyway...\n"); printf("Before stringprep (length %d): ", strlen(buf)); for (i=0; i < strlen(buf); i++) printf("%02x ", buf[i] & 0xFF); printf("\n"); rc = stringprep(buf, BUFSIZ, 0, stringprep_nameprep); if (rc != STRINGPREP_OK) printf("Stringprep failed with rc %d...\n", rc); else { printf("After stringprep (length %d): ", strlen(buf)); for (i=0; i < strlen(buf); i++) printf("%02x ", buf[i] & 0xFF); printf("\n"); } return 0; }
Libidn2
Libidn2 is an implementation of the IDNA2008 + TR46 specifications (RFC 5890, RFC 5891, RFC 5892, RFC 5893, TR 46). Libidn2 is a standalone library, without any dependency on Libidn. Libidn2 is believed to be a complete IDNA2008 / TR46 implementation, but has yet to be as extensively used as the original Libidn library.
Libidn2 uses GNU libunistring for Unicode processing and GNU libiconv for character set conversion.
Libidn2 can be downloaded from https://ftp.gnu.org/gnu/libidn/ [via HTTPS] and ftp://ftp.gnu.org/gnu/libidn/ [via FTP]. It can also be found on one of our FTP mirrors; please use a mirror if possible.
For documentation see the Libidn2 Manual web page, but quick links to the most popular formats:
- Libidn2 HTML Manual, generated by Texinfo
- API Reference, generated by GTK-DOC
For development, see the Libidn2 GitLab project page.
For Quality Assurance, we publish code coverage report and clang static analyzer output.
Initial development of Libidn2 has been sponsored by DENIC.