Few fixes regarding documentation
by Tomasz Rybak
Hello.
When building Debian packages, I noticed few issues. Those are not even
problems
(they do not prevent building) but some tools complain about them.
1. examples/download-examples-from-wiki.py
It has #! only as third line - first two are "from __future__ import ...".
Can you fix that?
2. pyopencl/scan.py contains copyright pointing to Thrust at
code.google.com.
While URL still contains source code, it is now archival - i.e. it has old
release, etc.
Would it be better to point to Thrust at GitHub:
https://github.com/thrust/thrust
(similarly for PyCUDA)
3. NDArray (from compyte) has some non-Python3 code:
byte-compiling
/home/admin/A/pyopencl-2016.1/debian/tmp/usr/lib/python3/dist-packages/pyope
ncl/compyte/ndarray/test_gpu_ndarray.py to test_gpu_ndarray.cpython-35.pyc
File
"usr/lib/python3/dist-packages/pyopencl/compyte/ndarray/test_gpu_ndarray.py",
line 314
print shp, dtype, offseted, order1, order2
^
SyntaxError: Missing parentheses in call to 'print'
byte-compiling
/home/admin/A/pyopencl-2016.1/debian/tmp/usr/lib/python3/dist-packages/pyope
ncl/compyte/ndarray/gen_elemwise.py to gen_elemwise.cpython-35.pyc
File
"usr/lib/python3/dist-packages/pyopencl/compyte/ndarray/gen_elemwise.py",
line 955
print sio.getvalue()
^
SyntaxError: invalid syntax
Again - nothing critical (only test and dead-function code), but prevents
pre-compiling Python3 code.
Thanks in advance for fixing.
Best regards.
Tomasz Rybak
2 years, 10 months
Re: [PyOpenCL] PyOpenCL linked to Beignet or PoCL
by Andreas Klöckner
Marmaduke,
I would also be interested in making PyOpenCL self contained enough that it can be installed without any further dependencies, in particular no OpenCL implementation. Ideally, 'pip install PyOpenCL' would just succeed no matter what. Pocl seems like a reasonable vehicle for that. The problem is that it has a nontrivial build system... And then there's LLVM... It feels like a substantial amount of engineering effort.
Another avenue I explored for a bit was a conda package. I even got to the point of making that work. I can dig those package recipes up if you're interested. I believe I also put them up on github. For those I did the thing that you asked about, which is link PyOpenCL directly to pocl.
Hope that helps, and let me know if you'd like to try and work together to engineer something.
Andreas
Am 18. Mai 2016 03:16:04 GMT-05:00, schrieb Marmaduke Woodman <mmwoodman(a)gmail.com>:
>hi
>
>I'm curious if anyone has succeeded in building and using PyOpenCL,
>linked
>directly to the Beignet or PortableCL runtimes?
>
>In my case, I'm writing a library using PyOpenCL, and it would be
>useful to
>fall-back on such an alternative, if the user of my library does not
>have
>an OpenCL installation such as Intel's or Nvidia's.
>
>
>Thanks,
>Marmaduke
>
>
>------------------------------------------------------------------------
>
>_______________________________________________
>PyOpenCL mailing list
>PyOpenCL(a)tiker.net
>https://lists.tiker.net/listinfo/pyopencl
2 years, 10 months
Non-blocking copy and wait-for event
by Schock, Jonathan
Hi all,
I am not quite getting the function of the event system together with
non-blocking copies.
I want to enqueue a non-blocking copy function which returns an event
for a kernel to wait on:
wev1 = cl.enqueue_copy(IOqueue, device_image, host_image,
is_blocking=False, origin=(0,0,0), region=host_image_shape)
program.kernel_name(workqueue,(work_group_shape,),None,device_image,wait_for
= [wev1])
Both queues are defined as OoO queues in the same context on the same
device.
In my profiling I can see the start of the kernel, before the copying is
finished. Does that mean, I have to use blocking copies,
or am I doing something else wrong?
Thanks for any help
Jonathan
2 years, 10 months
Installing pyopencl on RedHat 6
by Vizkelethy, Gyorgy
I am trying to install pyopencl on my Linux WS running RedHat 6.8. Its gcc is 4.4.7. The pyopencl installation failed because it could not find the atomic files. Which is correct, I could not find them either. It seems 4.4.7 does not support atomic yet. Is there an easy solution?
Thanks,
George
Gyorgy Vizkelethy
Sandia National Laboratories
Radiation-Solid Interactions
PO Box 5800, MS 1056
Albuquerque, NM 87185
USA
+1 (505) 284-3120
gvizkel(a)sandia.gov<mailto:gvizkel@sandia.gov>
2 years, 11 months
Re: [PyOpenCL] Pyopencl pickle support
by Andreas Kloeckner
Marcos Paulo Rocha <markao01(a)gmail.com> writes:
> But multithreading will suffer the problem of GIL doesn't it ?
Well, if you're doing it right, then all of your threads will either be
waiting for some OpenCL operation to complete or waiting for IO. Neither
of those is affected by the GIL.
Andreas
2 years, 11 months
Re: [PyOpenCL] Pyopencl pickle support
by Marcos Paulo Rocha
But multithreading will suffer the problem of GIL doesn't it ?
On Wed, Aug 17, 2016 at 7:33 PM, CRV§ADER//KY <crusaderky(a)gmail.com> wrote:
> So all you've got is disk io and opencl calls. You only need 1 process and
> some multithtreading.
>
> On 17 Aug 2016 19:03, "Marcos Paulo Rocha" <markao01(a)gmail.com> wrote:
>
>> In a application that i'm working, the programs is written in a form of a
>> graph. A example of part of one application is bellow:
>>
>> [image: Inline image 1]
>> Each node of the graph is a different process. When a node receive all of
>> your inputs, he is ready to start. This way, concurrency occurs naturally
>> between some nodes of the graph.
>> One of the goals of the library that i'm working, is to make easy,
>> development of applications with the behavior of the image. The ideia is
>> that nodes: Nodes CP_IN, CP_OUT and EX_KERNEL can be used to abstract the
>> copy, kernel setup and invocation to the final user.
>> I hope that now i have make myself clear about my goals and you can help
>> me to solve this problem.
>>
>> Thanks!
>> Best Regards,
>> Marcos Rocha
>>
>>
>> On Wed, Aug 17, 2016 at 1:35 PM, Andreas Kloeckner <
>> lists(a)informa.tiker.net> wrote:
>>
>>> Marcos Paulo Rocha <markao01(a)gmail.com> writes:
>>> > Thanks for reply Andreas.
>>> > Andreas, i need to access PyOpenCL objects in another process because
>>> i'm
>>> > working in a dataflow library and i would like to make copy and kernels
>>> > calls in parallel. I'm doing asynchronous copies and need that process
>>> > responsible for executing kernel receive event object of copy and the
>>> > buffer to set as kernel parameter. So there is another way to achieve
>>> this
>>> > behavior without using pickle ?
>>>
>>> I'm not sure what goal the different processes achieve here. To do
>>> concurrent copy and kernel invocations, all you need is two different
>>> commandqueues (from a single thread even). Just submit the copies to one
>>> and the kernel to the other. They'll run in parallel if the hardware is
>>> capable of doing that.
>>>
>>> Andreas
>>>
>>
>>
2 years, 11 months
Re: [PyOpenCL] Pyopencl pickle support
by CRV§ADER//KY
So all you've got is disk io and opencl calls. You only need 1 process and
some multithtreading.
On 17 Aug 2016 19:03, "Marcos Paulo Rocha" <markao01(a)gmail.com> wrote:
> In a application that i'm working, the programs is written in a form of a
> graph. A example of part of one application is bellow:
>
> [image: Inline image 1]
> Each node of the graph is a different process. When a node receive all of
> your inputs, he is ready to start. This way, concurrency occurs naturally
> between some nodes of the graph.
> One of the goals of the library that i'm working, is to make easy,
> development of applications with the behavior of the image. The ideia is
> that nodes: Nodes CP_IN, CP_OUT and EX_KERNEL can be used to abstract the
> copy, kernel setup and invocation to the final user.
> I hope that now i have make myself clear about my goals and you can help
> me to solve this problem.
>
> Thanks!
> Best Regards,
> Marcos Rocha
>
>
> On Wed, Aug 17, 2016 at 1:35 PM, Andreas Kloeckner <
> lists(a)informa.tiker.net> wrote:
>
>> Marcos Paulo Rocha <markao01(a)gmail.com> writes:
>> > Thanks for reply Andreas.
>> > Andreas, i need to access PyOpenCL objects in another process because
>> i'm
>> > working in a dataflow library and i would like to make copy and kernels
>> > calls in parallel. I'm doing asynchronous copies and need that process
>> > responsible for executing kernel receive event object of copy and the
>> > buffer to set as kernel parameter. So there is another way to achieve
>> this
>> > behavior without using pickle ?
>>
>> I'm not sure what goal the different processes achieve here. To do
>> concurrent copy and kernel invocations, all you need is two different
>> commandqueues (from a single thread even). Just submit the copies to one
>> and the kernel to the other. They'll run in parallel if the hardware is
>> capable of doing that.
>>
>> Andreas
>>
>
>
2 years, 11 months
Re: [PyOpenCL] Pyopencl pickle support
by Andreas Kloeckner
Marcos Paulo Rocha <markao01(a)gmail.com> writes:
> Thanks for reply Andreas.
> Andreas, i need to access PyOpenCL objects in another process because i'm
> working in a dataflow library and i would like to make copy and kernels
> calls in parallel. I'm doing asynchronous copies and need that process
> responsible for executing kernel receive event object of copy and the
> buffer to set as kernel parameter. So there is another way to achieve this
> behavior without using pickle ?
I'm not sure what goal the different processes achieve here. To do
concurrent copy and kernel invocations, all you need is two different
commandqueues (from a single thread even). Just submit the copies to one
and the kernel to the other. They'll run in parallel if the hardware is
capable of doing that.
Andreas
2 years, 11 months
Re: [PyOpenCL] Pyopencl pickle support
by Marcos Paulo Rocha
Thanks for reply Andreas.
Andreas, i need to access PyOpenCL objects in another process because i'm
working in a dataflow library and i would like to make copy and kernels
calls in parallel. I'm doing asynchronous copies and need that process
responsible for executing kernel receive event object of copy and the
buffer to set as kernel parameter. So there is another way to achieve this
behavior without using pickle ?
On Wed, Aug 17, 2016 at 4:58 AM, CRV§ADER//KY <crusaderky(a)gmail.com> wrote:
> It does make sense to send host-side buffers over the network though, or
> as a way to save a state to file.
> You could do the same with device buffers (with an auto copy from device
> memory when pickling and to device when unpicking) although you'd need to
> disambiguate which device you wish to unpickle to, maybe through a
> module-wide switch...
>
> On 17 Aug 2016 05:44, "Andreas Kloeckner" <lists(a)informa.tiker.net> wrote:
>
>> Hi Marcos,
>>
>> Marcos Paulo Rocha <markao01(a)gmail.com> writes:
>> > Hi, i'm using multiprocessing and pyopencl and need to pass pyopencl
>> > objects between processes. The problem is that pyopencl objects aren't
>> > compatibles with pickle module, using by multiprocessing lib. Any idea
>> to
>> > solve this problem ?
>>
>> Honestly, the only PyOpenCL object that I could think of that we could
>> meaningfully pickle is a device. Contexts, command queues, buffers, and
>> all the other stuff are not likely to make sense in the context of
>> another process. (At the OpenCL API level they're pointers, and they
>> each carry substantial non-enumerable state that we just can't package
>> up.) But even for devices, you're probably better off sending (platform
>> name, device name, index) triples...
>>
>> Sorry,
>> Andreas
>>
>> _______________________________________________
>> PyOpenCL mailing list
>> PyOpenCL(a)tiker.net
>> https://lists.tiker.net/listinfo/pyopencl
>>
>
2 years, 11 months
Re: [PyOpenCL] Pyopencl pickle support
by Andreas Kloeckner
"CRV§ADER//KY" <crusaderky(a)gmail.com> writes:
> It does make sense to send host-side buffers over the network though, or as
> a way to save a state to file.
> You could do the same with device buffers (with an auto copy from device
> memory when pickling and to device when unpicking) although you'd need to
> disambiguate which device you wish to unpickle to, maybe through a
> module-wide switch...
Well, you'd have to figure out what context to unpickle to--at which
point it's likely simpler to just pickle a numpy array with the data.
Andreas
2 years, 11 months
