Subject: Re: bug#75440: dejagnu does not like java.io.Console of OpenJDK >= 22

From: Chris Nospam <[email-masked]>
Date: Tue, 14 Jan 2025 11:12:01 +0000

Dear Jacob,

sorry for my late reply and thanks for your rapid and detailed answer. Also I am sorry for the faulty default.exp, it was a quick & dirty version to trigger the bug and only a short snipped of my real project. It should only show the problem with the first echo. However, I have enhanced it somehow now, see the attachement.

> Tcl interpolates backslash escapes in double-quoted strings

I changed now \r\n to \\r\\n in all regular expressions.

Please forgive me, I am not an expert in expect script but I try to do my best. I would be happy if you could give me some (direct) advices.

I will open a corresponding issue at https://bugreport.java.com and report back a link. Hopefully, they will look into/fix it and not only dismiss that using dejagnu is an edge case.

Many thanks again,
Chris

Gesendet: Donnerstag, 9. Januar 2025 um 02:55

Von: "Jacob Bachmeyer" <[email-masked]>

An: "Chris Nospam" <[email-masked]>, [email-masked]

Betreff: Re: bug#75440: dejagnu does not like java.io.Console of OpenJDK >= 22

On 1/8/25 10:44, Chris Nospam via Bug-dejagnu via wrote:

Dear maintainers,

starting with OpenJDK Java 22 java programs using java.io.Console cannot be testet with dejagnu any more. With JDK 21 it works like a charm.

This is a JVM bug or misfeature.  We have recently had similar problems with MinGW GCC tested under Wine. In that case, it was a consequence of Wine's new support for Windows quasi-ptys.

I belive it has to do with excape sequences that are not filtered out correctly or similar. I am using dejagnu 1.6.3-1, debain package under Ubuntu 24.04.

DejaGnu does not and cannot filter escape sequences.  Expect matches the patterns it is given.  You can write patterns in your testsuite that accommodate or even verify escape sequences from the program under test.

For an easy reproduction of the bug I have attached a simple example simple.zip. Just unpack it and execute exectest.sh. Be sure to update JAVA_HOME in this script to a jdk 22 or 23 installation.

In testsuite/simple.log some escape sequences are shown were no one are expected:
^[[?1h^[=^[[?2004hhello
^[[?1l^[>^[[?1000l^[[?2004lhello

The JVM IO facilities are inserting escape sequences without your program calling for them. This is a JVM bug or misfeature that Expect (on which DejaGnu is built) exposes.

The attached simple echo example works after simply commenting the lines 13-14 (using Console) and uncommenting lines 16-17 (using BufferedReader) in testsuite/simple.sh.
Alternatively it also works by passing -Djdk.console=java.base to the jvm (1st line in exectest.sh), to trigger the old implementation in JDK.

#!/usr/bin/env -S java --source 23
// #!/usr/local/java/jdk-23.0.1/bin/java --source 23 -Djdk.console=java.base

import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStreamReader;

public class ConsoleTest {

public static void main(String[] args) throws IOException {
System.out.print("prompt> ");

java.io.Console c = System.console();
String input = c.readLine();

// BufferedReader in = new BufferedReader(new InputStreamReader(System.in));
// String input = in.readLine();

System.out.println(input);
System.out.println("prompt> ");
}

}

First, think about what your program actually does compared to what your testsuite code expects it to do.  Examine the message produced at the end of the test run in simple.log.  In fact, that message is produced *after* echo.exp has completed.  Where does that message come from?

(You will benefit far more from the lesson if you find it for yourself than if I tell you.  There is only one place in your testsuite that can emit that message.)

Second, the inability to send text to the program under test is not a failure:  it is an error and probably an UNRESOLVED test.  A block that is executed as a result of catching a Tcl error probably should be invoking "error" or possibly "unresolved", not "fail" or even "pass" (unless you are testing an error condition).

Third, Tcl interpolates backslash escapes in double-quoted strings.  You will need to double (some of) the backslashes and/or review re_syntax(n) for Tcl regexp syntax.

Fourth, think about simple_test could actually fail.  That depends on what you are trying to do, but any "pass" should have a corresponding "fail" somewhere.

(Most of the above refers to Expect code in your testsuite/config/default.exp file.)

It has something to do with the change of the Java default Console provider to JLine in Java 22.
https://bugs.openjdk.org/browse/JDK-8309141is is not special with DejaGnu; we have recently had similar issues with
Wine, encountered while testing mingw32 builds on *nix hosts
However, as Java Console works everywhere else (windows/linux console/bash/EclipseIDE internal "console"/IntelliJ internal "console") as far I can see, it seems to be something special with dejagnu.

No, the problem is with the JVM and/or your programs and scripts.  It appears to work everywhere else because those escape sequences either perform terminal control functions or are ignored.  You can write regexps that will optionally match and discard the escape sequences.  Expect exposes this issue because it is strict about matching exactly what you tell it to match.

-- Jacob