Subject: bug#75440: dejagnu does not like java.io.Console of OpenJDK >= 22

From: Jacob Bachmeyer <[email-masked]>
Date: Wed, 8 Jan 2025 19:55:22 -0600

On 1/8/25 10:44, Chris Nospam via Bug-dejagnu via wrote:
> Dear maintainers,
>
> starting with OpenJDK Java 22 java programs using java.io.Console cannot be testet with dejagnu any more. With JDK 21 it works like a charm.

This is a JVM bug or misfeature.  We have recently had similar problems 
with MinGWGCC tested under Wine. In that case, it was a consequence of 
Wine's new support for Windows quasi-ptys.

> I belive it has to do with excape sequences that are not filtered out correctly or similar. I am using dejagnu 1.6.3-1, debain package under Ubuntu 24.04.

DejaGnu does not and cannot filter escape sequences.  Expect matches the 
patterns it is given.  You can write patterns in your testsuite that 
accommodate or even verify escape sequences from the program under test.

> For an easy reproduction of the bug I have attached a simple example simple.zip. Just unpack it and execute exectest.sh. Be sure to update JAVA_HOME in this script to a jdk 22 or 23 installation.
>
> In testsuite/simple.log some escape sequences are shown were no one are expected:
> ^[[?1h^[=^[[?2004hhello
> ^[[?1l^[>^[[?1000l^[[?2004lhello

The JVM IO facilities are inserting escape sequenceswithout your program 
calling for them. This is a JVM bug or misfeature that Expect (on which 
DejaGnu is built) exposes.

> The attached simple echo example works after simply commenting the lines 13-14 (using Console) and uncommenting lines 16-17 (using BufferedReader) in testsuite/simple.sh.
> Alternatively it also works by passing -Djdk.console=java.base to the jvm (1st line in exectest.sh), to trigger the old implementation in JDK.
>
> #!/usr/bin/env -S java --source 23
> // #!/usr/local/java/jdk-23.0.1/bin/java --source 23 -Djdk.console=java.base
>
> import java.io.BufferedReader;
> import java.io.IOException;
> import java.io.InputStreamReader;
>
> public class ConsoleTest {
>
>      public static void main(String[] args) throws IOException {
>          System.out.print("prompt> ");
>
>          java.io.Console c = System.console();
>          String input = c.readLine();
>
>          // BufferedReader in = new BufferedReader(new InputStreamReader(System.in));
>          // String input = in.readLine();
>
>          System.out.println(input);
>          System.out.println("prompt> ");
>      }
>
> }

First, think about what your program actually does compared to what your 
testsuite code expects it to do.  Examine the message produced at the 
end of the test run in simple.log.  In fact, that message is produced 
*after* echo.exp has completed.  Where does that message come from?

(You will benefit far more from the lesson if you find it for yourself 
than if I tell you.  There is only one place in your testsuite that can 
emit that message.)

Second, the inability to send text to the program under test is not a 
failure:  it is an error and probably an UNRESOLVED test.  A block that 
is executed as a result of catching a Tcl error probably should be 
invoking "error" or possibly "unresolved", not "fail" or even "pass" 
(unless you are testing an error condition).

Third, Tcl interpolates backslash escapes in double-quoted strings.  You 
will need to double (some of) the backslashes and/or review re_syntax(n) 
for Tcl regexp syntax.

Fourth, think about simple_test could actually fail.  That depends on 
what you are trying to do, but any "pass" should have a corresponding 
"fail" somewhere.

(Most of the above refers to Expect code in your 
testsuite/config/default.exp file.)

> It has something to do with the change of the Java default Console provider to JLine in Java 22.
> https://bugs.openjdk.org/browse/JDK-8309141is is not special with DejaGnu; we have recently had similar issues with
>   Wine, encountered while testing mingw32 builds on *nix hosts
> However, as Java Console works everywhere else (windows/linux console/bash/EclipseIDE internal "console"/IntelliJ internal "console") as far I can see, it seems to be something special with dejagnu.

No, the problem is with the JVM and/or your programs and scripts.  It 
appears to work everywhere else because those escape sequences either 
perform terminal control functions or are ignored.  You can write 
regexps that will optionally match and discard the escape sequences.  
Expect exposes this issue because it is strict about matching exactly 
what you tell it to match.

-- Jacob