Re: [PyOpenCL] PyOpenCL integration
by Andreas Kloeckner
Hi Rajan,
On Mon, 12 Sep 2011 16:57:30 -0500, "Yadav, Rajan1" <Rajan1.Yadav(a)amd.com> wrote:
> I am Rajan Yadav, a co-op at AMD, Austin. I am working towards
> developing a set of python wrappers for our existing OpenCL based FFT
> libraries. Our wrappers would be very thin and we want to make use of
> PyOpenCL to do stuff that OpenCL does in our existing libraries. For
> example, python user will himself create context and command queues
> and pass them on to our library. I am running into some problems in
> getting those PyOpenCL objects passed to our FFT library as C++
> objects which were passed to python by PyOpenCL.
>
> My mentor Kent contacted you a few days ago regarding this
> issue. Because of some other tasks, I had to defer working on these
> wrappers.
>
> In the mail attached, you suggested to build PyOpenCL with
> USE_SHIPPED_BOOST = False in siteconf.py file. Also, I adjusted the
> paths for boost headers and libraries in the same file and then did
> make and make install. One problem here is that in siteconf file, the
> name of boost python library is configured as
> boost_python-gcc43-mt. No such library gets installed when followed
> the procedure at http://wiki.tiker.net/BoostInstallationHowto
> . Instead, there is libboost_python.so in $HOME/pool/lib
> directory. So, I changed boost_python-gcc43-mt to boost_python in
> siteconf and moved forward, hoping that is right. Please correct me if
> its not.
That's correct. That default will also be changed in the next version of
PyOpenCL.
> The next suggestion that you gave was to include the header
> wrap_cl.hpp in our code. This implies that I should now be able to
> allocate the object of type pyopencl::context which I am able to. Now,
> we have a function in C++ that expects an object of type
> pyopencl::context along with bunch of other arguments. If we omit this
> pyopencl::context type argument, python is able call the c++ function
> successfully. But when python passes the "Context" type object to this
> function, there is a runtime ArgumentError:
>
> File "pointers.py", line 49, in <module>
> print "pyFFT,clAmdFftCreateDefaultPlan() = ", pyFFT.clAmdFftCreateDefaultPlan(plHandle, ctx1, i, 100)
> Boost.Python.ArgumentError: Python argument types in
> pyFFT.clAmdFftCreateDefaultPlan(pyClAmdFftPlanHandle, Context, clAmdFftDim, int)
> did not match C++ signature:
> clAmdFftCreateDefaultPlan(pyClAmdFftPlanHandle, pyopencl::context, clAmdFftDim, int)
>
> From what I understand, somewhere between python and our C++ code, the
> interface doesn't understand that "Context" type object in python is
> same as "pyopencl::context" type object and this where the problem
> arises. In your previous email you also said "('class buffer' is
> likely the one you need.)". I did not understand this point. Do
> pyopencl objects use different kind of memory allocation ? Why and at
> what point will I need buffer object in our wrappers ?
When you say 'Context', that object (let's call it 'obj') satisfies
'isinstance(obj, pyopencl.Context)'?
Another idea: Check using the 'ldd' tool whether the boost python
libraries used by your package and pyopencl are really the same:
Python 2.6.6 (r266:84292, Dec 26 2010, 22:31:48)
[GCC 4.4.5] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import pyopencl
>>> pyopencl._cl.__file__
'/home/andreas/src/pyopencl/pyopencl/_cl.so'
>>>
~ ldd /home/andreas/src/pyopencl/pyopencl/_cl.so
linux-vdso.so.1 => (0x00007fffadbbf000)
libboost_python-py26.so.1.46.1 => /usr/lib/libboost_python-py26.so.1.46.1 (0x00007f18690db000)
libOpenCL.so.1 => /home/andreas/pack/AMD-APP-SDK-v2.4-lnx64/lib/x86_64/libOpenCL.so.1 (0x00007f1868ed5000)
libstdc++.so.6 => /usr/lib/libstdc++.so.6 (0x00007f1868bcc000)
libm.so.6 => /lib/libm.so.6 (0x00007f186894a000)
libgcc_s.so.1 => /lib/libgcc_s.so.1 (0x00007f1868733000)
libpthread.so.0 => /lib/libpthread.so.0 (0x00007f1868517000)
libc.so.6 => /lib/libc.so.6 (0x00007f1868194000)
libutil.so.1 => /lib/libutil.so.1 (0x00007f1867f90000)
libdl.so.2 => /lib/libdl.so.2 (0x00007f1867d8c000)
librt.so.1 => /lib/librt.so.1 (0x00007f1867b84000)
/lib64/ld-linux-x86-64.so.2 (0x00007f1869653000)
(On Windows, Dependency Walker does something similar.)
Next--and probably the best idea by looking at your function
signature--try accepting a reference to the context instead of a
by-value call.
Hope some of this helps,
Andreas
8 years, 8 months
PyHPC2011 at SC11: Workshop on Python for High Performance and Scientific Computing
by Andreas Kloeckner
Hi all,
This might be of interest to some of you:
http://bit.ly/pyhpc2011
>From the web page:
The workshop aims at bringing together researchers and practitioners
from industry and academia using Python for all aspects of high
performance and scientific computing. The goal is to present
Python-based scientific applications and libraries, to discuss general
topics regarding the use of Python (such as language design and
performance issues), and to share experience using Python in scientific
computing education.
Paper submission deadline is September 19--so it's not too late to get
your contribution in.
Andreas
8 years, 8 months
Re: [PyOpenCL] Feature proposal: GPUArray views
by Andreas Kloeckner
On Fri, 9 Sep 2011 13:17:36 +1000, Bogdan Opanchuk <mantihor(a)gmail.com> wrote:
> Hello,
>
> I recently found myself missing a certain feature in PyCuda (and, in
> fact, the same can be said about PyOpenCL). Namely, the feature is a
> view of a part of GPUArray.
> For example, if I have some large array and want to perform some
> operation on the part of it, I have to introduce additional parameters
> like 'offset' and 'length':
>
> a = GPUArray(full_size, dtype)
> func(a, offset=1024, length=2048)
>
> In my opinion, this can be too error-prone, and this would be better:
>
> a = GPUArray(full_size, dtype)
> a_view = a.get_view(offset=1024, length=2048)
> func(a_view)
>
> Now func() does not have to worry about offsets and can just work with
> a_view as if it were a common GPUArray. This especially helps if
> func() is some library function which you cannot/do not want to mess
> with. Other advantage of views would be that you can reuse some large
> temporary array by creating different views to use at different steps
> (instead of creating many temporary arrays which take more memory).
>
> So, at this point I am just interested whether I am the only one who
> will find this feature useful. Unfortunately, in the nearest future I
> will not have time to add it myself. I am not even sure whether it
> should go to PyCuda/OpenCL or straight to compyte (or is it there
> already?).
PyCUDA already supports this--just do array[start:end]. PyOpenCL not
yet, but I'm meaning to add this at some point, unless compyte gets
there first.
Andreas
8 years, 9 months
Feature proposal: GPUArray views
by Bogdan Opanchuk
Hello,
I recently found myself missing a certain feature in PyCuda (and, in
fact, the same can be said about PyOpenCL). Namely, the feature is a
view of a part of GPUArray.
For example, if I have some large array and want to perform some
operation on the part of it, I have to introduce additional parameters
like 'offset' and 'length':
a = GPUArray(full_size, dtype)
func(a, offset=1024, length=2048)
In my opinion, this can be too error-prone, and this would be better:
a = GPUArray(full_size, dtype)
a_view = a.get_view(offset=1024, length=2048)
func(a_view)
Now func() does not have to worry about offsets and can just work with
a_view as if it were a common GPUArray. This especially helps if
func() is some library function which you cannot/do not want to mess
with. Other advantage of views would be that you can reuse some large
temporary array by creating different views to use at different steps
(instead of creating many temporary arrays which take more memory).
So, at this point I am just interested whether I am the only one who
will find this feature useful. Unfortunately, in the nearest future I
will not have time to add it myself. I am not even sure whether it
should go to PyCuda/OpenCL or straight to compyte (or is it there
already?).
Best regards,
Bogdan
8 years, 9 months
Intel sdk verbosity
by Vincent Favre-Nicolin
Hi,
I've been using PyCUDA a lot (see http://pynx.sf.net) and I am just
switching to PyOpenCL, which seems to work very nicely, including with CPUs.
Right now I am working under Ubuntu with amd+intel+nvidia sdk's, which seem
to work nicely along each other and PyOpenCL.
Only slightly annoying thing, the intel compiler seems to be a bit verbose
when it does not need to be, e.g. I get messages like:
##########
Build on <pyopencl.Device 'Intel(R) Core(TM)2 Quad CPU Q9550 @ 2.83GHz' at
0x1af72c0> succeeded, but said:
Build started
Kernel <Fhkl> was successfully vectorized
Done.
##########
This is a pretty minor issue, but does someone know how to turn off this
message from the intel compiler (while still keeping real warnings, I don't
want to completely turn off PyOpenCL reporting of messages). I tried looking in
the intel manual, but did not find a compiler option for that.
Incidentally, the intel compiler seems much more efficient (3.5x speed) than
AMD's for my Intel Core 2 Q9550. Not sure why, either better vectorization or
faster sin/cos functions.
regards,
--
Vincent Favre-Nicolin http://inac.cea.fr
CEA/Grenoble Institut Nanosciences & Cryogénie
Laboratoire SP2M/Nano-structures et Rayonnement Synchrotron
17, rue des Martyrs
38054 Grenoble Cedex 9 - France
Université Joseph Fourier http://www.ujf-grenoble.fr
tél: (+33) 4 38 78 95 40 fax: (+33) 4 38 78 51 38
8 years, 9 months