8000 Update to Java 11 breaks Pyserini · Issue #832 · castorini/anserini · GitHub
[go: up one dir, main page]
Skip to content

Update to Java 11 breaks Pyserini #832

New issue
New issue
Closed
Closed
Update to Java 11 breaks Pyserini#832

Description

@chriskamphuis
chriskamphuis
opened on Oct 22, 2019

When updating to Java 11 the path the JRE is not correctly configured for latest version of pyjnius that is installed with pip. This can be fixed by cloning the latest version from github and installing that version, but I assume this undesirable.

Activity

  1. lintool commented on Oct 22, 2019

    @lintool
    lintool
    on Oct 22, 2019
    Member

    hey @chriskamphuis any idea why src/test/python/python_interface.py didn't fail then? it's part of Travis CI definition...

  2. chriskamphuis commented on Oct 22, 2019

    @chriskamphuis
    chriskamphuis
    on Oct 22, 2019
    ContributorAuthor

    no clue .... it did not work for me locally because of this, so I assumed it was a problem in general, maybe it was my system specifically

  3. chriskamphuis commented on Oct 23, 2019

    @chriskamphuis
    chriskamphuis
    on Oct 23, 2019
    ContributorAuthor

    btw your Travis uses Python 2. Dont think that will be the reason, but might be nice to change it to Python 3.

  4. lintool commented on Oct 26, 2019

    @lintool
    lintool
    on Oct 26, 2019
    Member

    A bit more detail on this; I've replicated this bug. On macOS 10.14.6:

    (python36) iMac-Pro:anserini jimmylin$ conda list | grep pyjnius
pyjnius                   1.2.0                     <pip>

    The error I get is:

    >>> from pyserini.search import pysearch
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "./src/main/python/pyserini/search/pysearch.py", line 21, in <module>
    from ..pyclass import JSearcher, JString, JArrayList
  File "./src/main/python/pyserini/pyclass.py", line 28, in <module>
    from jnius import autoclass, cast
  File "/anaconda3/envs/python36/lib/python3.6/site-packages/jnius/__init__.py", line 13, in <module>
    from .reflect import *  # noqa
  File "/anaconda3/envs/python36/lib/python3.6/site-packages/jnius/reflect.py", line 15, in <module>
    class Class(with_metaclass(MetaJavaClass, JavaClass)):
  File "/anaconda3/envs/python36/lib/python3.6/site-packages/six.py", line 827, in __new__
    return meta(name, bases, d)
  File "jnius/jnius_export_class.pxi", line 114, in jnius.MetaJavaClass.__new__
  File "jnius/jnius_export_class.pxi", line 164, in jnius.MetaJavaClass.resolve_class
  File "jnius/jnius_env.pxi", line 11, in jnius.get_jnienv
  File "jnius/jnius_jvm_dlopen.pxi", line 90, in jnius.get_platform_jnienv
  File "jnius/jnius_jvm_dlopen.pxi", line 59, in jnius.create_jnienv
SystemError: Error calling dlopen(b'/Library/Java/JavaVirtualMachines/jdk-11.0.4.jdk/Contents/Home/jre/lib/server/libjvm.dylib': b'dlopen(/Library/Java/JavaVirtualMachines/jdk-11.0.4.jdk/Contents/Home/jre/lib/server/libjvm.dylib, 10): image not found'

    Issue is documented here.

    Indeed, the lib has moved:

    $ ls /Library/Java/JavaVirtualMachines/jdk-11.0.4.jdk/Contents/Home/lib/server/libjvm.dylib 
/Library/Java/JavaVirtualMachines/jdk-11.0.4.jdk/Contents/Home/lib/server/libjvm.dylib

    So a (janky) symlink will fix the issue:

    $ sudo mkdir -p /Library/Java/JavaVirtualMachines/jdk-11.0.4.jdk/Contents/Home/jre/lib/server/

$ ls /Library/Java/JavaVirtualMachines/jdk-11.0.4.jdk/Contents/Home/lib/server/libjvm.dylib 
/Library/Java/JavaVirtualMachines/jdk-11.0.4.jdk/Contents/Home/lib/server/libjvm.dylib

$ sudo ln -s /Library/Java/JavaVirtualMachines/jdk-11.0.4.jdk/Contents/Home/lib/server/libjvm.dylib /Library/Java/JavaVirtualMachines/jdk-11.0.4.jdk/Contents/Home/jre/lib/server/libjvm.dylib

  5. lintool commented on Nov 1, 2019

    @lintool
    lintool
    on Nov 1, 2019
    Member

    The colab demo doesn't seem to have this issue: https://colab.research.google.com/drive/1GOCyWyYW_fwJMKy5FVWdYHEC2IlvpqZN

    Edit: nevermind, above colab notebook doesn't use pyjnius.

  6. lintool commented on Nov 2, 2019

    @lintool
    lintool
    on Nov 2, 2019
    Member

    I just noticed, @Impavidity and @tuzhucheng mentioned the same issue (but different OS?) in #805. Unclear why colab works though...?

  7. lintool commented on Nov 2, 2019

    @lintool
    lintool
    on Nov 2, 2019
    Member

    Colab demo illustrating issue and fix: https://colab.research.google.com/drive/1r1pRq_BfWS486kg2qwVH5iBfbhK_GPCg

  8. lintool commented on Nov 2, 2019

    @lintool
    lintool
    on Nov 2, 2019
    Member

    Issue and resolution are documented in https://github.com/castorini/pyserini
    Closing issue.

  9. lintool
    closed this as completedon Nov 2, 2019
  10. crystina-z
    added a commit that references this issue on Oct 28, 2022

    Add script to verify (download) prebuilt indexes (castorini#832)

    104e70e
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions
      0