Opened 22 months ago

Closed 2 months ago

#615 closed defect (fixed)

problems finding module zope

Reported by: jerobarraco Owned by: htgoebel
Priority: high Milestone: PyInstaller 3.0
Component: PyInstaller Version: 2.0
Severity: major Keywords:
Cc:

Description (last modified by htgoebel)

my app depends on ZODB, and uses ZEO, that in turns depends on zope.event and zope.interface, which are two different eggs but use the same namespace.

site-packages /
    zope.event.xxxx.egg/
       zope/
          __init__.py
          event/
            __init__.py
    zope.interface.xxxx.egg/
       zope/
          __init__.py
          interface/
            __init__.py

the zope.event.xxx.egg/zope/init.py file contains :

__import__('pkg_resources').declare_namespace(__name__)

and zope.interface.xxx.egg/zope/init.py contains:

# this is a namespace package
try:
    import pkg_resources
    pkg_resources.declare_namespace(__name__)
except ImportError:
    import pkgutil
    __path__ = pkgutil.extend_path(__path__, __name__)

i tried to write a hook, but it can't pick "hook-zope.event.py"
i tried using hiddenimports and even tried to set the pathex
obviously the app works correctly when using from source. but when i try to package it (with pyinstaller) it says it can't find zope.event.
"matysek_" from irc recommended me to try modulegraph and file a report.
here's what i've done :

>>> from modulegraph.find_modules import find_modules as fm
>>> mg = fm(['banta/__init__.py'])
>>> f = open('o.txt', 'w')
>>> for i in mg.flatten(): f.write(str(i))
...
>>> f.close()

and it finds it

Change History (15)

comment:1 Changed 22 months ago by matysek

  • Milestone set to PyInstaller 2.1
  • Priority changed from normal to high

We could try to use modulegraph library for finding dependent modules.

comment:2 Changed 22 months ago by matysek

Example output of found dependencies by modulegraph: http://puu.sh/1b1dP

comment:3 Changed 16 months ago by matysek

  • Milestone changed from PyInstaller 2.1 to PyInstaller 2.2

comment:4 follow-up: Changed 11 months ago by glyph@…

This is apparently making it so that people can't use Twisted with pyinstaller. Is there any known workaround?

comment:5 Changed 11 months ago by htgoebel

  • Description modified (diff)

comment:6 Changed 11 months ago by htgoebel

  • Component changed from Hooks to PyInstaller
  • Milestone changed from PyInstaller 3.0 to PyInstaller 2.1
  • Severity changed from normal to major
  • Status changed from new to assigned

comment:7 Changed 11 months ago by htgoebel

Also see #502:

comment:8 Changed 11 months ago by htgoebel

  • Milestone changed from PyInstaller 2.1 to PyInstaller 3.0

Pushing to milestone PyInstaller? 3.0 again. Implementing this is complex. Since modulegraph says to support it, wait solving issue #439 should give it for free.

comment:9 in reply to: ↑ 4 Changed 2 months ago by anonymous

Replying to glyph@…:

This is apparently making it so that people can't use Twisted with pyinstaller. Is there any known workaround?

Yes, this is making is so people can't use Twisted w/ pyinstaller. At least on Windows.

comment:10 Changed 2 months ago by glyph@…

Would it be possible to work around this by having a generic distutils or setuptools recipe, that would just include all the modules that the setup.py says should be installed? I recommended something similar for py2app here: https://bitbucket.org/ronaldoussoren/py2app/issue/148/package-based-on-distutils-setuptools

comment:11 Changed 2 months ago by htgoebel

I again worked one over than hour on this issue. Namespace-packages are very hard to implement, since there are so many different kinds of implementations. I'm afraid, we will not implement this voluntary in PyInstaller 2.1 since we are moving forward to modulegraph. Sorry.

comment:12 follow-up: Changed 2 months ago by anonymous

modulegraph seems to be a step backwards. Just package the whole stdlib and let setup.py do its job with dependencies and data files; modulegraph is the reason that cloud deployments work fine in Python (virtualenv + wheels) and desktop deployments are a constant mess (py2app, py2exe, and pyinstaller all inherit the same mess of trying to guess what modules should be included, then failing)

comment:13 in reply to: ↑ 12 Changed 2 months ago by htgoebel

Replying to anonymous:

Just package the whole stdlib and let setup.py do its job with dependencies and data files

PyInstallers design idea is to *not* package the whole stdlib. But if you like, you can to this yourself, it should be very easy: Just put library.zip into the dist-directory and add it to sys.path. We'll happily include your recipe into the wiki.

But I doubt, including the whole stdlib would solve the problem, for two reasons:
a) zope.interface is not part of the stdlib
b) zope is a namespace package, which may not be easy to include even if using some workaround.

comment:14 Changed 2 months ago by matysek

Anyway, modulegraph is able to detect zope.interface package without any workarounds.

comment:15 Changed 2 months ago by htgoebel

  • Resolution set to fixed
  • Status changed from assigned to closed

I just implemented support for nspkg.pth-style namespaces as used by zope.interface in [2cf7654e35]. Please use the latest development version.

Note: See TracTickets for help on using tickets.