____________________________________________________________________________

   [1]http://invisible-island.net/
   Copyright © 2006-2018,2022 by Thomas E. Dickey
     ____________________________________________________________________________

     * [2](top)
     * [3]Synopsis
     * [4]History
     * [5]Metrics
     * [6]Platforms
     * [7]Bugs
     * [8]Changes
     * [9]Documentation
     * [10]Download
     * [11]Screenshots
     * [12]Related links
          + [13]Mailing list discussion
          + [14]Packages for Luit
          + [15]Other Luit sources
          + [16]Discussion of Luit usage
          + [17]Other programs

                 luit - Locale and ISO 2022 support for Unicode terminals

Synopsis

   Luit is a filter that can be run between an arbitrary application and a UTF-8
   terminal emulator. It will convert application output from the locale's encoding
   into UTF-8, and convert terminal input from UTF-8 into the locale's encoding. It
   is mainly used to support [18]xterm.

History

   Luit was written by Juliusz Chroboczek for the [19]XFree86 Project in 2001-2002.
   There were improvements and fixes by several people, in particular Tomohiro
   Kubota's extensions for CJK encodings.

   Like [20]xterm, luit uses the MIT-X11 license. There was some early speculation
   (in 2001) about using GPL which can be seen in mailing list archive ([21]this and
   [22]this for example). However, because GPL-licensed code is/was unacceptable in
   that realm, that speculation came to [23]nothing.

   There was no maintainer for some time; I adopted it in [24]2006 to ensure that it
   continued to support xterm (details are listed in the [25]luit.log.html file
   within the source).

   Besides the maintenance issue that attracted my attention in 2005 (untested
   changes to compiled-in file locations by Xorg hackers), Luit has had from the
   outset a technical issue: its associated font-encoding library.

   Juliusz Chroboczek used the font-encoding library to work around performance
   issues with direct use of iconv. This solution has proven to be a drawback:
     * the font-encoding library is little used (other than by luit), and also lacks
       a maintainer.
     * the font-encoding library does not provide the full range of encodings that
       iconv does.
     * the Xorg configure scripting and other dependencies surrounding the library
       have been subject to uncontrolled growth.

   I solved the problem by implementing an efficient conversion using iconv. Luit
   still supports the font-encoding library if it is found by the configure script.
   If you choose, luit can easily be built using iconv.

   However, as of luit 2.0, the font-encoding library has been deprecated:
     * Luit includes all of the relevant functionality for using the ".enc" files
       which are distributed separately. You may have these files as a separate
       package, e.g., "xfonts-encodings", or as part of "xfonts-x11-fonts-misc",
       "x11-font-encodings" or even "encodings". If you have trouble finding the
       package, look for a specific file such as adobe-standard.enc. The encoding
       files are rarely packaged with luit, and oddly enough are never made a
       package dependency. The only other use that I am aware of for the files is
       for the defunct xprint program.
       To see which ".enc" files luit may use, run

luit -list-fontenc

       [26]Here is sample output. The old version of luit can use only about a third
       of these encodings, i.e.,

     big5.eten-0, big5hkscs-0, dec-special, gb18030.2000-0, gb18030.2000-1,
     gb2312.1980-0, gbk-0, ibm-cp437, ibm-cp850, ibm-cp852, ibm-cp866, iso8859-11,
     iso8859-13, iso8859-16, jisx0201.1976-0, jisx0208.1990-0, jisx0212.1990-0,
     ksc5601.1987-0, microsoft-cp1250, microsoft-cp1251, microsoft-cp1252, tcvn-0
       With luit 2.0, the -encoding option permits you to use the remaining files
       (as well as any you may have customized):

     adobe-dingbats, adobe-standard, adobe-symbol, armscii-8, ascii-0, big5-0,
     big5.cp950-0, cns11643-1, cns11643-2, cns11643-3, gb18030-0, iso8859-6.16,
     iso8859-6.8x, jisx0208.1983-0, ksc5601.1992-3, ksx1001.1997-0, ksx1001.1998-0,
     ksx1001.1998-3, ksxjohab-1, microsoft-ansi, microsoft-cp1253,
     microsoft-cp1254, microsoft-cp1255, microsoft-cp1256, microsoft-cp1257,
     microsoft-cp1258, microsoft-win3.1, mulearabic-0, mulearabic-1, mulearabic-2,
     mulelao-1, sun.unicode.india-0, suneu-greek, tis620-0, tis620-2,
     tis620.2529-1, tis620.2533-0, tis620.2533-1, viscii1.1-1
     * Some of the ".enc" files are unused by the old luit because the font-encoding
       library has built-in tables of the ISO-8859-x encodings and a few others.
       With luit 2.0, you can make a list of the built-in tables as well as change
       luit's preference when looking in the font-encoding files, built-in tables
       and iconv tables.
     * Luit 2.0 can use the data from iconv directly without relying upon external
       ".enc" files. The ".enc" files (and built-in tables) are preferred for
       performance reasons. Existing users of luit would complain about the loss of
       1- or 2-tenths of a second for startup with CJK encodings. Really.
       Normally luit uses your locale settings to determine the corresponding
       character encoding. Use --list-iconv to see the available choices, e.g.,

luit -list-iconv

       [27]Here is sample output on a suitably configured system. Your system may
       have fewer (locale support generally has been made more difficult to
       configure in systems geared toward novice developers such as Ubuntu). But the
       portable iconv implementation does support a wide range of encodings, and you
       may find additional encodings using

iconv -l

       On the Debian system where I am writing this, that gives a list of 1168
       encodings.

Metrics

   Here is a summary of the code growth, starting from XFree86 and contrasting
   against changes made in the Xorg fork. To provide a better comparison, I first
   discarded generated or externally-maintained files:

     .gitignore, Makefile.in (when autogenerated), ChangeLog (when autogenerated),
     aclocal.m4 (when autogenerated), config.h.in (when autogenerated), autogen.sh,
     compile, config.guess, config.sub, configure, depcomp, install-sh, missing,
     mkdirs.sh, mkinstalldirs, builtin.c (generated), *.enc (generated)

   In short, discard the files which are not maintained by the respective developers
   of each branch. The autogenerated files were initially included, and later
   dropped without any effort expended.

   CAPTION: Luit code-growth

   Date Branch Release Total
   Files Total
   Lines Diffstat Diff against
   Files
   Changed Lines
   Added Lines
   Deleted
   2002-12-08 XFree86 snapshot 15 3490 N/A N/A N/A N/A
   2004-10-27 XFree86 snapshot 15 3780 8 403 113 2002-12-08
   2005-12-21 Xorg 1.0.1 21 3899 8 218 99 2004-10-27
   [28]2006-08-21 XTerm [29]20060820 20 6158 21 4044 1666 2004-10-27
   2006-11-08 Xorg [30]1.0.2 21 3915 4 62 46 1.0.1
   2008-01-31 Xorg [31]1.0.3 21 3918 9 108 105 1.0.2
   [32]2008-08-23 XTerm [33]20080823 20 6650 18 971 479 [34]2006-08-21
   [35]2008-08-29 XTerm 20080829 20 6714 3 108 44 [36]2008-08-23
   [37]2009-08-16 XTerm [38]20090816 21 7003 15 436 172 [39]2008-08-29
   2009-10-01 Xorg [40]1.0.4 21 3951 7 80 47 1.0.3
   [41]2009-10-14 XTerm [42]1.0-20091014 21 7080 8 149 47 [43]2009-08-16
   2010-02-09 Xorg [44]1.0.5 18 3973 3 36 14 1.0.4
   [45]2010-06-01 XTerm [46]1.0-20100601 32 7324 31 806 562 [47]2009-10-14
   [48]2010-06-05 XTerm [49]1.0-20100605 32 7515 8 206 15 [50]2010-06-01
   2010-10-06 Xorg [51]1.1.0 20 4653 17 2702 2022 1.0.5
   [52]2010-11-28 XTerm [53]1.0-20101128 37 8820 25 1651 346 [54]2010-06-05
   [55]2011-10-30 XTerm 1.2-20111030 37 9253 16 620 187 [56]2010-11-28
   [57]2012-01-29 XTerm [58]1.2-20120129 37 9688 11 549 114 [59]2011-10-30
   2012-03-23 Xorg [60]1.1.1 20 4655 8 39 37 1.1.0
   [61]2012-10-14 XTerm [62]1.2-20121014 37 9727 15 873 834 [63]2012-01-29
   [64]2013-02-17 XTerm [65]2.0-20130217 39 12586 21 3505 646 [66]2012-10-14
   [67]2014-12-04 XTerm 2.0-20141204 39 12702 10 196 80 [68]2013-02-17
   [69]2015-07-06 XTerm 2.0-20150706 39 12893 9 521 330 [70]2014-12-04
   [71]2016-05-08 XTerm 2.0-20160508 39 12980 10 144 57 [72]2015-07-06

   You may notice several things from the data:
     * I was not able to find an announcement for luit 1.0.1 release.
     * It appears (to me, at any rate) that my Freshmeat announcement in August 2006
       was the first luit announcement outside of XFree86 development.
     * My distribution is larger:
          + I write my own autoconf macros, and distribute those with the source.
          + I have included packaging scripts for the past couple of years.
     * I have done most of the development work on luit since 2004.
     * The large change in luit 1.1.0 is due to Xorg copying my work into their
       repository.
     * That Xorg commit used 1.0-20100601. Even though it was made nearly a month
       later, it omits the bug-fix which I made shortly after.

Platforms

   Luit is known to work with
     * SVR4-based Unix systems: IRIX64, HPUX, Solaris
     * Other Unix systems: AIX, Tru64
     * BSD-based systems: FreeBSD, NetBSD, OpenBSD, Mac OS X
     * Linux

   Aside from Solaris, luit had not been ported to other Unix systems before 2012.

   For some older versions (noticed with HPUX and FreeBSD) you may need to use the
   -p command-line option to work around a problem copying the terminal settings
   from the standard input to the pseudo-terminal.

Bugs

   You should report bugs to [73]me.
     ____________________________________________________________________________

Changes

   See the changelog for details:

     [74]changes with links...

Documentation

     * [75]luit program ([76]pdf) ([77]postscript) ([78]plain text)
     ____________________________________________________________________________

Download

     * [79]The source
     * [80]archives

Screenshots

   [81]Here are some screenshots of luit.

Related links

   There are numerous references on the net to luit. Here are a few of the more
   interesting ones:

  Mailing list discussions

     * [82][I18n] Call for testers: luit in XFree86 CVS
       2001-11-07 thread started by Juliusz Chroboczek
     * [83][I18n][patch] xterm to invoke luit
       2002-01-30 thread discussing Tomohiro Kubota proposal to integrate xterm and
       luit.
     * [84][I18n][call for comments] XTerm patch to invoke luit
       2002-06-06 thread discussing Tomohiro Kubota patch to integrate xterm and
       luit.
     * [85][I18n]Test implementation of luit extension
       2002-06-25 thread discussing Tomohiro Kubota proposal to extend luit to
       handle CJK encodings.
     * [86][PATCH luit] Integrate changes from Thomas Dickey's luit-20100601 fork.
       Relevant discussion:
          + [87]luit forked? (April 2009 thread)
            Coopersmith's comments were made in the context of a different thread;
            the only purpose served was to widen the dispute:
          + [88]Documentation? (April 2009 thread)
          + [89]ANN luit 20100601 (June 2010)
          + [90]ANN luit 20100605 (June 2010)
          + [91][ANNOUNCE] luit 1.1.0 (October 2010)
          + [92]ANN: luit 20101128 (November 2010)

  Packages for Luit

     * [93]OSWatershed.org (archived in 2014)
     * [94]NetBSD and [95]OpenSUSE have packages of luit 2.x, for example.
     * The older (Xorg) versions are typically bundled with the "xorg-x11-apps"
       (RPM) or [96]"x11-utils" (DPKG) packages.
     * Luit is actually not an X application, although it is most useful with xterm.
       Some packagers needlessly link it against the X libraries.
       Luit depends on the "font-encoding" library, which has no X dependencies.
     * Some packages are incomplete, probably due to its inclusion in a miscellany
       of small X applications.
       For example, Red Hat RPMs omit the data files needed to support CJK
       encodings.
       In Debian, that is the [97]xfonts-encodings package. See these for the
       missing parts:
          + [98]git repository
          + [99]encodings tar-files.

  Other Luit sources

     * [100]Juliusz Chroboczek's page discussing luit
     * [101]Tomohiro Kubota page discussing luit and xterm
     * [102]XFree86 CVS (shows development work until 2006)
     * [103]Xorg git-repository (incorporates my changes through 2010-06-01)
     * [104]Xorg latest git repository is marked as an inactive project,
       recommending this site.

  Discussion of Luit usage

     * [105]terminal - working with filenames in a different encoding over ssh
     * [106]How to use grep/ack with files in arbitrary encoding?
     * [107]How to switch charset in terminal?

  Other programs

     * [108]cocot - COde COnverter on Tty
       Less portable and well-integrated, but essentially the same concept.
          + [109]#638838 - ITP: cocot -- Wraps and converts charset encoding of
            program input/output.
          + [110]Re: package description : cocot
            Jonathan Nieder comments on the similarity of luit and cocot, suggesting
            that luit's description would be a good starting point for the latter.
     * [111]Mosh: the mobile shell
       Mosh is more closely related to luit and cocot than to quasi-terminal
       emulators such as screen and tmux. It aims to enforce a particular style of
       UTF-8 encoding.
       For instance, Keith Winstein [112]comments that mosh does not support
       ISO-2022 locking shifts whereas screen and luit do this.
     * [113]jacot 0.2.3 : Python Package Index
       This references both luit and cocot.
     * [114]"tail -f | iconv -fjis" does not output anything
       discussion which proposes luit as solution, and mentions ttyconv (which uses
       iconv).

References

   1. https://invisible-island.net/
   2. https://invisible-island.net/luit/luit.html
   3. https://invisible-island.net/luit/#synopsis
   4. https://invisible-island.net/luit/#history
   5. https://invisible-island.net/luit/#metrics
   6. https://invisible-island.net/luit/#platforms
   7. https://invisible-island.net/luit/#bugs
   8. https://invisible-island.net/luit/#changes
   9. https://invisible-island.net/luit/#documentation
  10. https://invisible-island.net/luit/#download
  11. https://invisible-island.net/luit/#screenshot
  12. https://invisible-island.net/luit/#related
  13. https://invisible-island.net/luit/#related_discussion
  14. https://invisible-island.net/luit/#related_packages
  15. https://invisible-island.net/luit/#related_sources
  16. https://invisible-island.net/luit/#related_comments
  17. https://invisible-island.net/luit/#related_programs
  18. https://invisible-island.net/xterm/xterm.html
  19. http://www.xfree86.org/current/luit.1.html
  20. https://invisible-island.net/xterm/xterm.faq.html
  21. http://comments.gmane.org/gmane.comp.internationalization.linux/2814
  22. https://www.mail-archive.com/linux-utf8@nl.linux.org/msg01493.html
  23. http://osdir.com/ml/internationalization.linux/2001-07/msg00045.html
  24. http://comments.gmane.org/gmane.linux.lfs.beyond.devel/14643
  25. https://invisible-island.net/luit/luit.log.html
  26. https://invisible-island.net/luit/manpage/list-fontenc.txt
  27. https://invisible-island.net/luit/manpage/list-iconv.txt
  28. https://invisible-island.net/luit/luit.log.html#t20080821
  29. http://freecode.com/projects/luit/releases/234457
  30. http://www.spinics.net/lists/xorg/msg19270.html
  31. http://lists.x.org/archives/xorg-announce/2008-January/000446.html
  32. https://invisible-island.net/luit/luit.log.html#t20080823
  33. http://freecode.com/projects/luit/releases/283564
  34. https://invisible-island.net/luit/luit.log.html#t20080821
  35. https://invisible-island.net/luit/luit.log.html#t20080829
  36. https://invisible-island.net/luit/luit.log.html#t20080823
  37. https://invisible-island.net/luit/luit.log.html#t20090816
  38. http://freecode.com/projects/luit/releases/304017
  39. https://invisible-island.net/luit/luit.log.html#t20080829
  40. http://lists.x.org/archives/xorg-announce/2009-October/001080.html
  41. https://invisible-island.net/luit/luit.log.html#t20091014
  42. http://freecode.com/projects/luit/releases/306766
  43. https://invisible-island.net/luit/luit.log.html#t20090816
  44. http://lists.x.org/archives/xorg-announce/2010-February/001254.html
  45. https://invisible-island.net/luit/luit.log.html#t20100601
  46. http://freecode.com/projects/luit/releases/317555
  47. https://invisible-island.net/luit/luit.log.html#t20091014
  48. https://invisible-island.net/luit/luit.log.html#t20100605
  49. http://freecode.com/projects/luit/releases/317768
  50. https://invisible-island.net/luit/luit.log.html#t20100601
  51. http://lists.x.org/archives/xorg-announce/2010-October/001428.html
  52. https://invisible-island.net/luit/luit.log.html#t20101128
  53. http://freecode.com/projects/luit/releases/325122
  54. https://invisible-island.net/luit/luit.log.html#t20100605
  55. https://invisible-island.net/luit/luit.log.html#t20111030
  56. https://invisible-island.net/luit/luit.log.html#t20101128
  57. https://invisible-island.net/luit/luit.log.html#t20120129
  58. http://freecode.com/projects/luit/releases/337929
  59. https://invisible-island.net/luit/luit.log.html#t20111030
  60. http://lists.x.org/archives/xorg-announce/2012-March/001894.html
  61. https://invisible-island.net/luit/luit.log.html#t20121014
  62. http://freecode.com/projects/luit/releases/348937
  63. https://invisible-island.net/luit/luit.log.html#t20120129
  64. https://invisible-island.net/luit/luit.log.html#t20130217
  65. http://freecode.com/projects/luit/releases/352338
  66. https://invisible-island.net/luit/luit.log.html#t20121014
  67. https://invisible-island.net/luit/luit.log.html#t20141204
  68. https://invisible-island.net/luit/luit.log.html#t20130217
  69. https://invisible-island.net/luit/luit.log.html#t20150706
  70. https://invisible-island.net/luit/luit.log.html#t20141204
  71. https://invisible-island.net/luit/luit.log.html#t20160508
  72. https://invisible-island.net/luit/luit.log.html#t20150706
  73. mailto:dickey@invisible-island.net
  74. https://invisible-island.net/luit/luit.log.html
  75. https://invisible-island.net/luit/manpage/luit.html
  76. https://invisible-island.net/luit/manpage/luit.pdf
  77. https://invisible-island.net/luit/manpage/luit.ps
  78. https://invisible-island.net/luit/manpage/luit.txt
  79. https://invisible-island.net/datafiles/release/luit.tar.gz
  80. https://invisible-island.net/archives/luit/
  81. https://invisible-island.net/luit/luit-figures.html
  82. http://marc.info/?t=100515510000008&r=1&w=3
  83. http://www.mail-archive.com/i18n@xfree86.org/msg00531.html
  84. http://marc.info/?t=102337972200001&r=1&w=4
  85. http://marc.info/?t=102496973500003&r=1&w=2
  86. http://lists.x.org/archives/xorg-devel/2010-June/009697.html
  87. http://lists.x.org/archives/xorg/2009-April/thread.html#44897
  88. http://lists.x.org/archives/xorg/2009-April/thread.html#44661
  89. http://lists.x.org/archives/xorg/2010-June/thread.html#50128
  90. http://lists.x.org/archives/xorg/2010-June/050202.html
  91. http://lists.x.org/archives/xorg/2010-October/051226.html
  92. http://lists.x.org/archives/xorg/2010-November/thread.html#51792
  93. https://web.archive.org/web/20140717075110/http://oswatershed.org/pkg/luit
  94. http://cdn.netbsd.org/pub/pkgsrc/current/pkgsrc/x11/luit/README.html
  95. https://software.opensuse.org/package/luit
  96. http://packages.debian.org/search?keywords=x11-utils
  97. http://packages.debian.org/search?keywords=xfonts-encodings
  98. http://cgit.freedesktop.org/xorg/font/encodings
  99. http://xorg.freedesktop.org/releases/individual/font/
 100. http://www.pps.univ-paris-diderot.fr/~jch/software/luit
 101. http://www8.plala.or.jp/tkubota1/xterm.html
 102. http://cvsweb.xfree86.org/cvsweb/xc/programs/luit
 103. http://cgit.freedesktop.org/xorg/app/luit/
 104. https://gitlab.freedesktop.org/xorg/app/luit
 105. http://unix.stackexchange.com/questions/25350/working-with-filenames-in-a-different-encoding-over-ssh
 106. http://unix.stackexchange.com/questions/33574/how-to-use-grep-ack-with-files-in-arbitrary-encoding
 107. http://www.linuxquestions.org/questions/linux-software-2/how-to-switch-charset-in-terminal-925945/
 108. http://vmi.jp/software/cygwin/cocot.html
 109. http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=638838
 110. http://lists.debian.org/debian-l10n-english/2010/04/msg00032.html
 111. http://mosh.mit.edu/
 112. https://github.com/keithw/mosh/issues/115
 113. http://pypi.python.org/pypi/jacot/0.2.3
 114. http://unix.stackexchange.com/questions/35662/tail-f-iconv-fjis-does-not-output-anything