18 May
2013
18 May
'13
6:56 p.m.
Hi.
I am interested in getting PyOpenCL to work with PyPy, an
implementation of cpython with a JITwww.pypy.org . Has there been any
discussion or thought about doing this? PyPy has a basic
implementation of numpy called numpypy that I contribute to, and it
has a rudimentary numpy-compatible c interface available as an
external module at
https://bitbucket.org/antocuni/numpypy_c
The PyPy team has a cpython-compatible replacement for ctypes called
cffi, that is jit-friendly on PyPy and no slower than ctypes on
cpython.
So it seems like all the pieces exist to start, is anyone else interested in
getting the work done?
Or are there blocking issues I do not understand?
Matti
18 May
18 May
11:54 p.m.
Hi Matti,
Matti Picus <matti(a)picus.org.il> writes:
I am interested in getting PyOpenCL to work with PyPy,
an
implementation of cpython with a JITwww.pypy.org . Has there been any
discussion or thought about doing this? PyPy has a basic
implementation of numpy called numpypy that I contribute to, and it
has a rudimentary numpy-compatible c interface available as an
external module at
https://bitbucket.org/antocuni/numpypy_c
The PyPy team has a cpython-compatible replacement for ctypes called
cffi, that is jit-friendly on PyPy and no slower than ctypes on
cpython.
So it seems like all the pieces exist to start, is anyone else interested in
getting the work done?
I've been thinking about replacing the Boost.Python bits in PyOpenCL
with cffi, and I'd be interested in figuring out how feasible all this
is. It certainly looks doable from my point of view. My main objective
in getting PyPy to work with PyOpenCL is making kernel invocation even
cheaper than it currently is, or rather, be able to afford more
kernel-invocation-time niceties than we currently can.
I'd obviously like to preserve PyOpenCL's documented interface.
Andreas
20 May
20 May
6:15 p.m.
I prefer top-posting, let me know if it's a no-no.
It seems the way to proceed would be by creating another parallel
low-level interface to the system calls via cffi without pyboost, and
then slowly replace the existing low-level interfaces with the new ones
until we can eventually remove the old ones. Hopefully that will not
cause too many memory leaks in the transition.
How is the coverage of the test suite, can we be reasonably confident
that if tests pass and the benchmark does not suffer, the package still
is functional?
Matti
On 05/19/2013 02:54 AM, Andreas Kloeckner wrote:
Hi Matti,
Matti Picus <matti(a)picus.org.il> writes:
I am interested in getting PyOpenCL to work with
PyPy, an
implementation of cpython with a JITwww.pypy.org . Has there been any
discussion or thought about doing this? PyPy has a basic
implementation of numpy called numpypy that I contribute to, and it
has a rudimentary numpy-compatible c interface available as an
external module at
https://bitbucket.org/antocuni/numpypy_c
The PyPy team has a cpython-compatible replacement for ctypes called
cffi, that is jit-friendly on PyPy and no slower than ctypes on
cpython.
So it seems like all the pieces exist to start, is anyone else interested in
getting the work done?
I've been thinking about replacing the Boost.Python bits in PyOpenCL
with cffi, and I'd be interested in figuring out how feasible all this
is. It certainly looks doable from my point of view. My main objective
in getting PyPy to work with PyOpenCL is making kernel invocation even
cheaper than it currently is, or rather, be able to afford more
kernel-invocation-time niceties than we currently can.
I'd obviously like to preserve PyOpenCL's documented interface.
Andreas
11:10 p.m.
Matti Picus <matti(a)picus.org.il> writes:
It seems the way to proceed would be by creating
another parallel
low-level interface to the system calls via cffi without pyboost, and
then slowly replace the existing low-level interfaces with the new ones
until we can eventually remove the old ones. Hopefully that will not
cause too many memory leaks in the transition.
Since many of the objects interact, we'd also have to build an interface
layer between the boost-python wrappers and the cffi ones--which I'd
rather avoid.
IOW, I'm not sure it makes sense to have both in the same tree. I'd say
let's start slow by just wrapping the bare necessities from Context,
CommandQueue, Buffer, Program, and Kernel. (while taking notes on what
was left out) That should give us a proof-of-concept idea of how
difficult (and successful) this is all going to be.
How is the coverage of the test suite, can we be
reasonably confident
that if tests pass and the benchmark does not suffer, the package still
is functional?
Coverage is good. If it passes, it's in releasable state.
Andreas
19 Aug
19 Aug
12:02 p.m.
Hi
I am attending the London pypy sprint next week and looking for things
to work on. Has any work been done yet on porting pyopencl to cffi?
Best, Marko
On 05/21/2013 01:10 AM, Andreas Kloeckner wrote:
Matti Picus <matti(a)picus.org.il> writes:
It seems the way to proceed would be by creating
another parallel
low-level interface to the system calls via cffi without pyboost, and
then slowly replace the existing low-level interfaces with the new ones
until we can eventually remove the old ones. Hopefully that will not
cause too many memory leaks in the transition.
Since many of the objects interact,
we'd also have to build an interface
layer between the boost-python wrappers and the cffi ones--which I'd
rather avoid.
IOW, I'm not sure it makes sense to have both in the same tree. I'd say
let's start slow by just wrapping the bare necessities from Context,
CommandQueue, Buffer, Program, and Kernel. (while taking notes on what
was left out) That should give us a proof-of-concept idea of how
difficult (and successful) this is all going to be.
How is the coverage of the test suite, can we be
reasonably confident
that if tests pass and the benchmark does not suffer, the package still
is functional?
Coverage is good. If it passes, it's in releasable state.
Andreas
_______________________________________________
PyOpenCL mailing list
PyOpenCL(a)tiker.net
http://lists.tiker.net/listinfo/pyopencl
3:11 p.m.
Hi Marko,
marko <mbencun(a)gmail.com> writes:
I am attending the London pypy sprint next week and
looking for things
to work on. Has any work been done yet on porting pyopencl to cffi?
Matti Picus (whom you cc'd) said at one point that he was interested in
working on this. Not sure how far he got, but that's all I'm aware
of. I'm very supportive of the idea--if there's anything I can do to
help you along, let me know!
Andreas
24 Aug
24 Aug
6:53 p.m.
On 08/19/2013 06:11 PM, Andreas Kloeckner wrote:
Hi Marko,
marko writes:
I started a branch on github,
https://github.com/mattip/pyopencl/tree/cffi , to do just this, but
rapidly got bogged down by the extensive use of #define macros which I
tried turning into python function decorators, all this before getting
anything really to run. Perhaps another collaborator on this would make
it go faster.
Matti
I am attending the London pypy sprint next week
and looking for things
to work on. Has any work been done yet on porting pyopencl to cffi?
Matti Picus (whom you cc'd) said at one point that he was interested in
working on this. Not sure how far he got, but that's all I'm aware
of. I'm very supportive of the idea--if there's anything I can do to
help you along, let me know!
Andreas
2516
days inactive
2614
days old
6 comments
3 participants
participants (3)
-
Andreas Kloeckner -
marko -
Matti Picus