There must be some blocking / dead-locking 
problem in term.
Original Findings
# w | grep [t]sch
tschwing p1 192.168.10.60: Tue 8PM 0:03 2172 /bin/bash
tschwing p2 192.168.10.60: Tue 4PM 40hrs 689 emacs
tschwing p3 192.168.10.60: 8:52PM 11:37 15307 /bin/bash
tschwing p0 192.168.10.60: 6:42PM 11:47 8104 /bin/bash
tschwing p8 192.168.10.60: 8:27AM 0:02 16510 /bin/bash
Now open a new screen window, or login shell, or...
# ps -Af | tail
[...]
tschwinge 16538 676 p6 0:00.08 /bin/bash
root 16554 128 co 0:00.09 ps -Af
root 16555 128 co 0:00.01 tail
bash is started (on p6), but newer makes it to the shell promt; doesn't
even start to execute .bash_profile / .bashrc. The next shell started, on
the next available pseudoterminal, will work without problems.
The term on p6 has already been running before:
# ps -Af | grep [t]typ6
root 6871 3 - 5:45.86 /hurd/term /dev/ptyp6 pty-master /dev/ttyp6
In this situation, w will sometimes report erroneous values for IDLE
for the process using that terminal.
Killed that term instance, and things were fine again.
All this reproducible happens while running the GDB testsuite.
Have a freshly started shell blocking on such a term instance.
$ ps -F hurd-long -p 1766 -T -Q
PID TH# UID PPID PGrp Sess TH Vmem RSS %CPU User System Args
1766 0 3 1 1 6 131M 1.14M 0.0 0:28.85 5:40.91 /hurd/term /dev/ptyp3 pty-master /dev/ttyp3
0 0.0 0:05.76 1:08.48
1 0.0 0:00.00 0:00.01
2 0.0 0:06.40 1:11.52
3 0.0 0:05.76 1:09.89
4 0.0 0:05.42 1:06.74
5 0.0 0:05.50 1:04.25
... and after 5:45 h:
$ ps -F hurd-long -p 21987 -T -Q
PID TH# UID PPID PGrp Sess TH Vmem RSS %CPU User System Args
21987 1001 676 21987 21987 2 148M 2.03M 0.0 0:00.02 0:00.07 /bin/bash
0 0.0 0:00.02 0:00.07
1 0.0 0:00.00 0:00.00
$ ps -F hurd-long -p 1766 -T -Q
PID TH# UID PPID PGrp Sess TH Vmem RSS %CPU User System Args
1766 0 3 1 1 6 131M 1.14M 0.0 0:29.04 5:42.38 /hurd/term /dev/ptyp3 pty-master /dev/ttyp3
0 0.0 0:05.76 1:08.48
1 0.0 0:00.00 0:00.01
2 0.0 0:06.41 1:11.90
3 0.0 0:05.82 1:10.28
4 0.0 0:05.52 1:07.06
5 0.0 0:05.52 1:04.63
$ sudo gdb /hurd/term 1766
[sudo] password for tschwinge:
GNU gdb (GDB) 7.0-debian
Copyright (C) 2009 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "i486-gnu".
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>...
Reading symbols from /hurd/term...Reading symbols from /usr/lib/debug/hurd/term...done.
(no debugging symbols found)...done.
Attaching to program `/hurd/term', pid 1766
[New Thread 1766.1]
[New Thread 1766.2]
[New Thread 1766.3]
[New Thread 1766.4]
[New Thread 1766.5]
[New Thread 1766.6]
Reading symbols from /lib/libhurdbugaddr.so.0.3...Reading symbols from /usr/lib/debug/lib/libhurdbugaddr.so.0.3...
[System doesn't respond anymore, but no kernel crash.]
The very same behavior is still observable as of 2011-03-24.
Next: rebooted; on console started root shell, screen, a few spare windows; as
user started GDB test suite, noticed the PTY it's using; in a root shell
started GDB (the system one, for .debug stuff) on /hurd/term, set
noninvasive on, attach to the term that GDB is using.
2012-11-05
Log file from a 2011-09-07 run:
[...]
Running ../../../master/gdb/testsuite/gdb.base/readline.exp ...
spawn [...]/gdb/testsuite/../../gdb/gdb -nw -nx -data-directory [...]/gdb/testsuite/../data-directory
GNU gdb (GDB) 7.3.50.20110906-cvs
Copyright (C) 2011 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "i686-unknown-gnu0.3".
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
(gdb) set height 0
(gdb) set width 0
(gdb) dir
Reinitialize source path to empty? (y or n) y
Source directories searched: $cdir:$cwd
(gdb) dir ../../../master/gdb/testsuite/gdb.base
Source directories searched: [...]/gdb/testsuite/../../../master/gdb/testsuite/gdb.base:$cdir:$cwd
(gdb) p 1
$1 = 1
PASS: gdb.base/readline.exp: Simple operate-and-get-next - send p 1
(gdb) p 2
$2 = 2
PASS: gdb.base/readline.exp: Simple operate-and-get-next - send p 2
(gdb) p 3
$3 = 3
PASS: gdb.base/readline.exp: Simple operate-and-get-next - send p 3
(gdb) p 3(gdb) p 3PASS: gdb.base/readline.exp: Simple operate-and-get-next - C-p to p 3
^H2(gdb) p 2PASS: gdb.base/readline.exp: Simple operate-and-get-next - C-p to p 2
^H1(gdb) p 1PASS: gdb.base/readline.exp: Simple operate-and-get-next - C-p to p 1
^OFAIL: gdb.base/readline.exp: Simple operate-and-get-next - C-o for p 1
FAIL: gdb.base/readline.exp: operate-and-get-next with secondary prompt - send if 1 > 0
FAIL: gdb.base/readline.exp: print 42 (timeout)
FAIL: gdb.base/readline.exp: arrow keys with secondary prompt (timeout)
spawn [...]/gdb/testsuite/../../gdb/gdb -nw -nx -data-directory [...]/gdb/testsuite/../data-directory
ERROR: (timeout) GDB never initialized after 10 seconds.
ERROR: no fileid for coulomb
ERROR: no fileid for coulomb
UNRESOLVED: gdb.base/readline.exp: Simple operate-and-get-next - send p 7
testcase ../../../master/gdb/testsuite/gdb.base/readline.exp completed in 646 seconds
Running ../../../master/gdb/testsuite/gdb.base/wchar.exp ...
Executing on host: gcc -c -g -o [...]/gdb/testsuite/gdb.base/wchar0.o ../../../master/gdb/testsuite/gdb.base/wchar.c (timeout = 300)
spawn gcc -c -g -o [...]/gdb/testsuite/gdb.base/wchar0.o ../../../master/gdb/testsuite/gdb.base/wchar.c
Executing on host: gcc [...]/gdb/testsuite/gdb.base/wchar0.o -g -lm -o [...]/gdb/testsuite/gdb.base/wchar (timeout = 300)
spawn gcc [...]/gdb/testsuite/gdb.base/wchar0.o -g -lm -o [...]/gdb/testsuite/gdb.base/wchar
get_compiler_info: gcc-4-6-1
spawn [...]/gdb/testsuite/../../gdb/gdb -nw -nx -data-directory [...]/gdb/testsuite/../data-directory
ERROR: (timeout) GDB never initialized after 10 seconds.
ERROR: no fileid for coulomb
ERROR: no fileid for coulomb
ERROR: no fileid for coulomb
ERROR: couldn't load [...]/gdb/testsuite/gdb.base/wchar into [...]/gdb/testsuite/../../gdb/gdb (timed out).
ERROR: no fileid for coulomb
ERROR: Delete all breakpoints in delete_breakpoints (timeout)
ERROR: no fileid for coulomb
UNRESOLVED: gdb.base/wchar.exp: setting breakpoint at wchar.c:34 (timeout)
testcase ../../../master/gdb/testsuite/gdb.base/wchar.exp completed in 797 seconds
[...]
IRC, freenode, #hurd, 2012-08-09
In context of the select issue.
<braunr> i wonder where the tty allocation is made
<braunr> it could simply be that current applications don't handle old BSD
ptys correctly
<braunr> hm no, allocation is fine
<braunr> does someone know why there is no term instance for /dev/ttypX ?
<braunr> showtrans says "/hurd/term /dev/ttyp0 pty-slave /dev/ptyp0" though
<youpi> braunr: /dev/ttypX share the same translator with /dev/ptypX
<braunr> youpi: but how ?
<youpi> see the main function of term
<youpi> it attaches itself to the other node
<youpi> with file_set_translator
<youpi> just like pfinet can attach itself to /servers/socket/26 too
<braunr> youpi: isn't there a possible race when the same translator tries
to sets itself on several nodes ?
<youpi> I don't know
<tschwinge> There is.
<braunr> i guess it would just faikl
<braunr> fail
<tschwinge> I remember some discussion about this, possibly in context of
the IPv6 project.
<braunr> gdb shows weird traces in term
<braunr> i got this earlier today: http://www.sceen.net/~rbraun/gdb.txt
<braunr> 0x805e008 is the ptyctl, the trivs control for the pty
<tschwinge> braunr: How do you mean »weird«?
<braunr> tschwinge: some peropen (po) are never destroyed
<tschwinge> Well, can't they possibly still be open?
<braunr> they shouldn't
<braunr> that's why term doesn't close cleany, why select still reports
readiness, and why screen loops on it
<braunr> (and why each ssh session uses a different pty)
<tschwinge> ... but only on darnassus, I think? (I think I haven't seen
this anywhere else.)
<braunr> really ?
<braunr> i had it on my virtual machines too
<tschwinge> But perhaps I've always been rebooting systems quickly enough
to not notice.
<tschwinge> OK, I'll have a look next time I boot mine.
<braunr> i suppose it's why you can't login anymore quickly when syslog is
running
<braunr> i've traced the problem to ptyio.c, where pty_open_hook returns
EBUSY because ptyopen is still true
<braunr> ptyopen remains true because pty_po_create_hook doesn't get called
<youpi> tschwinge: I've seen the pty issue on exodar too, and on my qemu
image too
<braunr> err, pty_po_destroy_hook
<tschwinge> OK.
<braunr> and pty_po_destroy_hook doesn't get called from users.c because
po->cntl != ptyctl
<braunr> which means, somehow, the pty never gets closed
<youpi> oddly enough it seems to happen on all qemu systems I have, and no
xen system I have
<braunr> Oo
<braunr> are they all (xen and qemu) up to date ?
<braunr> (so we can remove versions as a factor)
<tschwinge> Aha. I only hve Xen and real hardware.
<youpi> braunr: no
<braunr> youpi: do you know any obscur site about ptys ? :)
<youpi> no
<youpi> well, actually yes
<youpi> http://dept-info.labri.fr/~thibault/a (in french)
<braunr> :D
<braunr> http://www.linusakesson.net/programming/tty/index.php looks
interesting
<youpi> indeed
IRC, freenode, #hurdfr, 2012-08-09
<braunr> youpi: ce que j'ai le plus de mal à comprendre, c'est ce qu'est un
"controlling tty"
<youpi> c'est le plus obscur d'obscur :)
<braunr> s'il est exclusif à une appli, comment ça doit se comporter sur un
fork, etc..
<youpi> de manière simple, c'est ce qui permet de faire ^C
<braunr> eh oui, et c'est sûrement là que ça explose
<youpi> c'est pas exclusif, c'est hérité
<braunr>
http://homepage.ntlworld.com/jonathan.deboynepollard/FGA/bernstein-on-ttys/cttys.html
IRC, freenode, #hurd, 2012-08-10
<braunr> youpi: and just to be sure about the test procedure, i log on a
system, type tty, see e.g. ttyp0, log out, and in again, then tty returns
ttyp1, etc..
<youpi> yes
<braunr> youpi: and an open (e.g. cat) on /dev/ptyp0 returns EBUSY
<youpi> indeed
<braunr> so on xen it doesn't
<braunr> grmbl
<youpi> I've never seen it, more precisely
<braunr> i also have the problem with a non-accelerated qemu
<braunr> antrik: do you have the term problems we've seen on your bare
hardware ?
<antrik> I'm not sure what problem you are seeing exactly :-)
<braunr> antrik: when logging through ssh, tty first returns ttyp0, and the
second time (after logging out from the first session) ttyp1
<braunr> antrik: and term servers that have been used are then stuck in a
busy state
<antrik> braunr: my ptys seem to be reused just fine
<braunr> or perhaps they didn't have the bug
<braunr> antrik: that's so weird
<antrik> (I do *sometimes* get hanging ptys, but that's a different issue
-- these are *not* busy; they just hang when reused...)
<braunr> antrik: yes i saw that too
<antrik> braunr: note though that my hurd package is many months old...
<antrik> (in fact everything on this system)
<braunr> antrik: i didn't see anything relevant about the term server in
years
<braunr> antrik: what shell do you use ?
<antrik> yeah, but such errors could be caused by all kinds of changes in
other parts of the Hurd, glibc, whatever...
<antrik> bash
IRC, freenode, #hurd, 2012-12-27
<youpi> we however have a similar symptom with screen
<youpi> shells don't terminate
<braunr> yes
<youpi> or at least the window doesn't close
<braunr> the screen problem is the same as the term servers not being properly closed
<youpi> k
<braunr> that one is still on my todo list
<braunr> and not easy
<youpi> like so many small items on the TODO lists :)
<braunr> that one is an important one :)
<braunr> because we're still using legacy pty, the number of terms is
limited
<braunr> which means at some point we can't log in any more using them
<braunr> (i regularly kill pty terms on darnassus to avoid that)
<braunr> it prevents screen and rsyslogd iirc from working correctly, which
is very annoying
<braunr> there may be other issues
Formal Verification
This issue may be a simple programming error, or it may be more complicated.
Methods of formal verification should be applied to confirm that there is
no error in /hurd/term's logic itself. There are tools for formal
verification/code analysis that can likely help here.
There is a FOSS Factory bounty (p277) on this task.