-
Notifications
You must be signed in to change notification settings - Fork 5
/
clx_extensions.html
402 lines (381 loc) · 16.6 KB
/
clx_extensions.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
<?xml version="1.0" encoding="utf-8"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" lang="en" xml:lang="en">
<head>
<!-- 01/28/2024 -->
<meta http-equiv="Content-Type" content="text/html;charset=utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1" />
<title> X11, Common Lisp, CLX and international input</title>
<meta name="author" content="David Bjergaard" />
<meta name="generator" content="Org Mode" />
<style>
#content { max-width: 60em; margin: auto; }
.title { text-align: center;
margin-bottom: .2em; }
.subtitle { text-align: center;
font-size: medium;
font-weight: bold;
margin-top:0; }
.todo { font-family: monospace; color: red; }
.done { font-family: monospace; color: green; }
.priority { font-family: monospace; color: orange; }
.tag { background-color: #eee; font-family: monospace;
padding: 2px; font-size: 80%; font-weight: normal; }
.timestamp { color: #bebebe; }
.timestamp-kwd { color: #5f9ea0; }
.org-right { margin-left: auto; margin-right: 0px; text-align: right; }
.org-left { margin-left: 0px; margin-right: auto; text-align: left; }
.org-center { margin-left: auto; margin-right: auto; text-align: center; }
.underline { text-decoration: underline; }
#postamble p, #preamble p { font-size: 90%; margin: .2em; }
p.verse { margin-left: 3%; }
pre {
border: 1px solid #e6e6e6;
border-radius: 3px;
background-color: #f2f2f2;
padding: 8pt;
font-family: monospace;
overflow: auto;
margin: 1.2em;
}
pre.src {
position: relative;
overflow: auto;
}
pre.src:before {
display: none;
position: absolute;
top: -8px;
right: 12px;
padding: 3px;
color: #555;
background-color: #f2f2f299;
}
pre.src:hover:before { display: inline; margin-top: 14px;}
/* Languages per Org manual */
pre.src-asymptote:before { content: 'Asymptote'; }
pre.src-awk:before { content: 'Awk'; }
pre.src-authinfo::before { content: 'Authinfo'; }
pre.src-C:before { content: 'C'; }
/* pre.src-C++ doesn't work in CSS */
pre.src-clojure:before { content: 'Clojure'; }
pre.src-css:before { content: 'CSS'; }
pre.src-D:before { content: 'D'; }
pre.src-ditaa:before { content: 'ditaa'; }
pre.src-dot:before { content: 'Graphviz'; }
pre.src-calc:before { content: 'Emacs Calc'; }
pre.src-emacs-lisp:before { content: 'Emacs Lisp'; }
pre.src-fortran:before { content: 'Fortran'; }
pre.src-gnuplot:before { content: 'gnuplot'; }
pre.src-haskell:before { content: 'Haskell'; }
pre.src-hledger:before { content: 'hledger'; }
pre.src-java:before { content: 'Java'; }
pre.src-js:before { content: 'Javascript'; }
pre.src-latex:before { content: 'LaTeX'; }
pre.src-ledger:before { content: 'Ledger'; }
pre.src-lisp:before { content: 'Lisp'; }
pre.src-lilypond:before { content: 'Lilypond'; }
pre.src-lua:before { content: 'Lua'; }
pre.src-matlab:before { content: 'MATLAB'; }
pre.src-mscgen:before { content: 'Mscgen'; }
pre.src-ocaml:before { content: 'Objective Caml'; }
pre.src-octave:before { content: 'Octave'; }
pre.src-org:before { content: 'Org mode'; }
pre.src-oz:before { content: 'OZ'; }
pre.src-plantuml:before { content: 'Plantuml'; }
pre.src-processing:before { content: 'Processing.js'; }
pre.src-python:before { content: 'Python'; }
pre.src-R:before { content: 'R'; }
pre.src-ruby:before { content: 'Ruby'; }
pre.src-sass:before { content: 'Sass'; }
pre.src-scheme:before { content: 'Scheme'; }
pre.src-screen:before { content: 'Gnu Screen'; }
pre.src-sed:before { content: 'Sed'; }
pre.src-sh:before { content: 'shell'; }
pre.src-sql:before { content: 'SQL'; }
pre.src-sqlite:before { content: 'SQLite'; }
/* additional languages in org.el's org-babel-load-languages alist */
pre.src-forth:before { content: 'Forth'; }
pre.src-io:before { content: 'IO'; }
pre.src-J:before { content: 'J'; }
pre.src-makefile:before { content: 'Makefile'; }
pre.src-maxima:before { content: 'Maxima'; }
pre.src-perl:before { content: 'Perl'; }
pre.src-picolisp:before { content: 'Pico Lisp'; }
pre.src-scala:before { content: 'Scala'; }
pre.src-shell:before { content: 'Shell Script'; }
pre.src-ebnf2ps:before { content: 'ebfn2ps'; }
/* additional language identifiers per "defun org-babel-execute"
in ob-*.el */
pre.src-cpp:before { content: 'C++'; }
pre.src-abc:before { content: 'ABC'; }
pre.src-coq:before { content: 'Coq'; }
pre.src-groovy:before { content: 'Groovy'; }
/* additional language identifiers from org-babel-shell-names in
ob-shell.el: ob-shell is the only babel language using a lambda to put
the execution function name together. */
pre.src-bash:before { content: 'bash'; }
pre.src-csh:before { content: 'csh'; }
pre.src-ash:before { content: 'ash'; }
pre.src-dash:before { content: 'dash'; }
pre.src-ksh:before { content: 'ksh'; }
pre.src-mksh:before { content: 'mksh'; }
pre.src-posh:before { content: 'posh'; }
/* Additional Emacs modes also supported by the LaTeX listings package */
pre.src-ada:before { content: 'Ada'; }
pre.src-asm:before { content: 'Assembler'; }
pre.src-caml:before { content: 'Caml'; }
pre.src-delphi:before { content: 'Delphi'; }
pre.src-html:before { content: 'HTML'; }
pre.src-idl:before { content: 'IDL'; }
pre.src-mercury:before { content: 'Mercury'; }
pre.src-metapost:before { content: 'MetaPost'; }
pre.src-modula-2:before { content: 'Modula-2'; }
pre.src-pascal:before { content: 'Pascal'; }
pre.src-ps:before { content: 'PostScript'; }
pre.src-prolog:before { content: 'Prolog'; }
pre.src-simula:before { content: 'Simula'; }
pre.src-tcl:before { content: 'tcl'; }
pre.src-tex:before { content: 'TeX'; }
pre.src-plain-tex:before { content: 'Plain TeX'; }
pre.src-verilog:before { content: 'Verilog'; }
pre.src-vhdl:before { content: 'VHDL'; }
pre.src-xml:before { content: 'XML'; }
pre.src-nxml:before { content: 'XML'; }
/* add a generic configuration mode; LaTeX export needs an additional
(add-to-list 'org-latex-listings-langs '(conf " ")) in .emacs */
pre.src-conf:before { content: 'Configuration File'; }
table { border-collapse:collapse; }
caption.t-above { caption-side: top; }
caption.t-bottom { caption-side: bottom; }
td, th { vertical-align:top; }
th.org-right { text-align: center; }
th.org-left { text-align: center; }
th.org-center { text-align: center; }
td.org-right { text-align: right; }
td.org-left { text-align: left; }
td.org-center { text-align: center; }
dt { font-weight: bold; }
.footpara { display: inline; }
.footdef { margin-bottom: 1em; }
.figure { padding: 1em; }
.figure p { text-align: center; }
.equation-container {
display: table;
text-align: center;
width: 100%;
}
.equation {
vertical-align: middle;
}
.equation-label {
display: table-cell;
text-align: right;
vertical-align: middle;
}
.inlinetask {
padding: 10px;
border: 2px solid gray;
margin: 10px;
background: #ffffcc;
}
#org-div-home-and-up
{ text-align: right; font-size: 70%; white-space: nowrap; }
textarea { overflow-x: auto; }
.linenr { font-size: smaller }
.code-highlighted { background-color: #ffff00; }
.org-info-js_info-navigation { border-style: none; }
#org-info-js_console-label
{ font-size: 10px; font-weight: bold; white-space: nowrap; }
.org-info-js_search-highlight
{ background-color: #ffff00; color: #000000; font-weight: bold; }
.org-svg { }
</style>
<link rel="stylesheet" type="text/css" href="style.css" />
</head>
<body>
<div id="content" class="content">
<h1 class="title"> X11, Common Lisp, CLX and international input</h1>
<div id="org2eeeead" class="figure">
<p><img src="./images/stumpwm-logo-stripe.png" alt="stumpwm-logo-stripe.png" width="600px" />
</p>
</div>
<div class="org" id="orgfb0a370">
<table border="2" cellspacing="0" cellpadding="6" rules="groups" frame="hsides">
<colgroup>
<col class="org-left" />
<col class="org-left" />
<col class="org-left" />
<col class="org-left" />
<col class="org-left" />
<col class="org-left" />
</colgroup>
<tbody>
<tr>
<td class="org-left"><a href="./index.html">Home</a></td>
<td class="org-left"><a href="screenshots.html">Screenshots</a></td>
<td class="org-left"><a href="documentation.html">Documentation</a></td>
<td class="org-left"><a href="download.html"> Download</a></td>
<td class="org-left"><a href="community.html">Community</a></td>
<td class="org-left"><a href="links.html">Links</a></td>
</tr>
</tbody>
</table>
</div>
<ul class="org-ul">
<li>Started writing: <span class="timestamp-wrapper"><span class="timestamp"><2018-05-10 Thu> </span></span></li>
<li>Posted: <span class="timestamp-wrapper"><span class="timestamp"><2018-05-11 Fri></span></span></li>
<li>Estimated time to read: 8 min</li>
</ul>
<div id="outline-container-org1ddb4b7" class="outline-2">
<h2 id="org1ddb4b7">TLDR;</h2>
<div class="outline-text-2" id="text-org1ddb4b7">
<p>
Common Lisp applications that use the X11 protocol are missing
important X11 extensions that would allow them to support
internationalized input. Specifically, the xkeyboard protocol must be
fully implemented to rectify these problems for any application
utlizing CLX.
</p>
</div>
</div>
<div id="outline-container-org33739c2" class="outline-2">
<h2 id="org33739c2">Introduction</h2>
<div class="outline-text-2" id="text-org33739c2">
<p>
In 1987, the X11 standard was formalized. It is the de-facto standard
for graphics primitives on the *nix desktop. X11 is a client-server
protocol that abstracts away direct calls to hardware rendering. It
allows <a href="https://en.wikipedia.org/wiki/X_Window_System">graphical desktops</a> to be rendered in mainframe environments
where the server is physically separated from the client. If you've
ever forwarded X11 over ssh, you've interacted with a remote X11
server. Most of the time the X11 server is resident on the local
machine, and the client-server connection is over a Unix socket.
</p>
<p>
It is important to understand that almost all user facing applications
are clients to the server (all the way down to your window manager).
This means that if you write an X11 client with a library that doesn't
support the modern features of your server, that application will not
take full advantage of the server. Other applications written with
different X11 client libraries which support modern features will have
no trouble responding to the new extensions in the appropriate manner.
</p>
<p>
It may sound like I'm harping on a subtle distinction that would never
come up in the real world. However it has in the case of
internationalization and localization support in the Common Lisp X
interface (CLX). Further, there seems to be no resources out there for
eager contributors to read and understand the problem from end to end.
This article aims to rectify that.
</p>
</div>
</div>
<div id="outline-container-orgbce6969" class="outline-2">
<h2 id="orgbce6969">CLX, Xlib and XCB</h2>
<div class="outline-text-2" id="text-orgbce6969">
<p>
For many years the standard library implementing X11 client protocols
in C was <a href="https://en.wikipedia.org/wiki/Xlib">Xlib</a>. In fact it is still widely used by higher level
tool-kits that provide further GUI abstractions. Xlib has its own set
of problems and cruft associated with its long shelf-life. In <a href="https://en.wikipedia.org/wiki/XCB">2001 XCB
(X C Bindings)</a> was developed to address these problems. XCB is unique
in that it defines the X protocol (and its extensions) in XML format
and then generates the C bindings dynamically. If you write an X11
client application in C, in 2018, you should be using XCB.
</p>
<p>
Most non-C languages interface with Xlib through some sort of foreign
function interface. This has the advantage that features added to Xlib
are carried over to languages which interface from it. The situation
for Common Lisp is different. The Common Lisp X interface (CLX) does
not interface with Xlib, instead it implements the protocol directly.
I'm speculating, but I expect that this exists because of the <a href="https://en.wikipedia.org/wiki/Lisp_machine">Lisp
Machines</a> of the 70s and 80s. They were entirely different animals from
Unix, and had Lisp baked into their hardware. In that environment, X11
would have been a protocol to allow a lisp machine to accept clients
from Unix mainframes. It makes sense then that the library would be
written in Common Lisp from the ground up.
</p>
<p>
What would happen next is still causing echos today in modern lisp
applications. In the time period of 1987-1993 the Lisp Machine market
collapsed in the wake of the <a href="https://en.wikipedia.org/wiki/AI_winter">AI winter</a>. The CLX manual, dated 1988,
came at the end of the Lisp Machine's relevance in computing research.
In <a href="https://en.wikipedia.org/wiki/History_of_Linux">1991 Linus Torvalds released</a> the first version of Linux, and
ushered in an era of unix-like desktops. The ubiquity of a free
version of unix-like operating system and the explosion of the
internet created market pressures which lead to internationalization
support in Xlib that didn't exist in the Common Lisp ecosystem.
Extensions such as xkeyboard (XKB) have never been fully implemented,
and it is difficult as a sole developer to fully implement a protocol
standard as complicated as XKB. This is further complicated by the
fact that end users typically want to be able to type non-english
letters and it is not clear how to communicate this in a way that
points developers to the XKB extension.
</p>
</div>
</div>
<div id="outline-container-org93cfee8" class="outline-2">
<h2 id="org93cfee8">The state of the art</h2>
<div class="outline-text-2" id="text-org93cfee8">
<p>
There has been significant progress. SBCL supports unicode characters,
and <a href="https://filonenko-mikhail.github.io/clx-truetype/">rendering</a> toolkits can <a href="https://github.com/dbjergaard/clx-ft2-renderer">render</a> the glyphs appropriately. The
missing secret sauce is a implementation of XKB. A few people have
ventured deep enough into this problem to identify the missing piece,
but a full solution has not made it into <a href="https://github.com/sharplispers/clx">sharplispers/clx</a>, the
community's source for CLX.
</p>
<p>
McClim (a major graphics toolkit) has placed a significant bounty on
this <a href="https://github.com/McCLIM/McCLIM/issues/35">very issue</a>. The success condition for that bounty has been that
"input [is obeyed by a] keyboard layout set with <code>setxkbmap</code>
asynchronusly at runtime by the user from another process (i.e
terminal)." Another more <a href="https://github.com/filonenko-mikhail/clx-xkeyboard">promising effort</a> is available as a standalone
extension in quicklisp. I have just discovered this code, and I have
no idea how close it is to solving the issue.
</p>
</div>
</div>
<div id="outline-container-orgffbff53" class="outline-2">
<h2 id="orgffbff53">Contributing</h2>
<div class="outline-text-2" id="text-orgffbff53">
<p>
If you've made it this far, you're probably eagerly consuming any
information on this topic and want to contribute. My nominal plan is
to host a fork of <a href="https://github.com/sharplispers/clx">sharplispers/clx</a>. I will contact Mikhail Filonenko
about taking over his copy of xkeyboard and I will add stubs of
function definitions for xkeyboard. Contributing will be a matter of
opening a PR with an implementation of a missing function. Once we
have the majority of the protocol implemented we can contribute it
back to sharplispers/clx and the whole Lisp community will profit.
</p>
<p>
Even if you are not affected by internationalized input, having XKB
available will be useful as it makes it possible to do things that are
impossible with the core implementation of keyboards. Specifically, it
becomes possible to have multiple keys trigger the same key event
which makes it possible to have multiple prefix keys in StumpWM. It
also makes it possible to support weird keyboard layouts that have
different modifiers than what a generic keyboard can handle.
</p>
</div>
</div>
<div id="outline-container-orge8e7e53" class="outline-2">
<h2 id="orge8e7e53">Conclusion</h2>
<div class="outline-text-2" id="text-orge8e7e53">
<p>
Hopefully this will serve as a roadmap of how and why Common Lisp has
poor support for international input and will help dispel some of the
confusion around the topic. If you are new to Common Lisp and affected
by this issue, take a look at my fork of CLX. There may be something
you can contribute to which will help us solve the problem once and
for all.
</p>
</div>
</div>
</div>
</body>
</html>