PyOpenCL September 2011

pyopencl@tiker.net

14 participants
25 discussions

by Andreas Kloeckner

Hi Rajan, On Mon, 12 Sep 2011 16:57:30 -0500, "Yadav, Rajan1" <Rajan1.Yadav(a)amd.com> wrote: > I am Rajan Yadav, a co-op at AMD, Austin. I am working towards > developing a set of python wrappers for our existing OpenCL based FFT > libraries. Our wrappers would be very thin and we want to make use of > PyOpenCL to do stuff that OpenCL does in our existing libraries. For > example, python user will himself create context and command queues > and pass them on to our library. I am running into some problems in > getting those PyOpenCL objects passed to our FFT library as C++ > objects which were passed to python by PyOpenCL. > > My mentor Kent contacted you a few days ago regarding this > issue. Because of some other tasks, I had to defer working on these > wrappers. > > In the mail attached, you suggested to build PyOpenCL with > USE_SHIPPED_BOOST = False in siteconf.py file. Also, I adjusted the > paths for boost headers and libraries in the same file and then did > make and make install. One problem here is that in siteconf file, the > name of boost python library is configured as > boost_python-gcc43-mt. No such library gets installed when followed > the procedure at http://wiki.tiker.net/BoostInstallationHowto > . Instead, there is libboost_python.so in $HOME/pool/lib > directory. So, I changed boost_python-gcc43-mt to boost_python in > siteconf and moved forward, hoping that is right. Please correct me if > its not. That's correct. That default will also be changed in the next version of PyOpenCL. > The next suggestion that you gave was to include the header > wrap_cl.hpp in our code. This implies that I should now be able to > allocate the object of type pyopencl::context which I am able to. Now, > we have a function in C++ that expects an object of type > pyopencl::context along with bunch of other arguments. If we omit this > pyopencl::context type argument, python is able call the c++ function > successfully. But when python passes the "Context" type object to this > function, there is a runtime ArgumentError: > > File "pointers.py", line 49, in <module> > print "pyFFT,clAmdFftCreateDefaultPlan() = ", pyFFT.clAmdFftCreateDefaultPlan(plHandle, ctx1, i, 100) > Boost.Python.ArgumentError: Python argument types in > pyFFT.clAmdFftCreateDefaultPlan(pyClAmdFftPlanHandle, Context, clAmdFftDim, int) > did not match C++ signature: > clAmdFftCreateDefaultPlan(pyClAmdFftPlanHandle, pyopencl::context, clAmdFftDim, int) > > From what I understand, somewhere between python and our C++ code, the > interface doesn't understand that "Context" type object in python is > same as "pyopencl::context" type object and this where the problem > arises. In your previous email you also said "('class buffer' is > likely the one you need.)". I did not understand this point. Do > pyopencl objects use different kind of memory allocation ? Why and at > what point will I need buffer object in our wrappers ? When you say 'Context', that object (let's call it 'obj') satisfies 'isinstance(obj, pyopencl.Context)'? Another idea: Check using the 'ldd' tool whether the boost python libraries used by your package and pyopencl are really the same: Python 2.6.6 (r266:84292, Dec 26 2010, 22:31:48) [GCC 4.4.5] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import pyopencl >>> pyopencl._cl.__file__ '/home/andreas/src/pyopencl/pyopencl/_cl.so' >>> ~ ldd /home/andreas/src/pyopencl/pyopencl/_cl.so linux-vdso.so.1 => (0x00007fffadbbf000) libboost_python-py26.so.1.46.1 => /usr/lib/libboost_python-py26.so.1.46.1 (0x00007f18690db000) libOpenCL.so.1 => /home/andreas/pack/AMD-APP-SDK-v2.4-lnx64/lib/x86_64/libOpenCL.so.1 (0x00007f1868ed5000) libstdc++.so.6 => /usr/lib/libstdc++.so.6 (0x00007f1868bcc000) libm.so.6 => /lib/libm.so.6 (0x00007f186894a000) libgcc_s.so.1 => /lib/libgcc_s.so.1 (0x00007f1868733000) libpthread.so.0 => /lib/libpthread.so.0 (0x00007f1868517000) libc.so.6 => /lib/libc.so.6 (0x00007f1868194000) libutil.so.1 => /lib/libutil.so.1 (0x00007f1867f90000) libdl.so.2 => /lib/libdl.so.2 (0x00007f1867d8c000) librt.so.1 => /lib/librt.so.1 (0x00007f1867b84000) /lib64/ld-linux-x86-64.so.2 (0x00007f1869653000) (On Windows, Dependency Walker does something similar.) Next--and probably the best idea by looking at your function signature--try accepting a reference to the context instead of a by-value call. Hope some of this helps, Andreas

9 years, 4 months

PyHPC2011 at SC11: Workshop on Python for High Performance and Scientific Computing

by Andreas Kloeckner

Hi all, This might be of interest to some of you: http://bit.ly/pyhpc2011 >From the web page: The workshop aims at bringing together researchers and practitioners from industry and academia using Python for all aspects of high performance and scientific computing. The goal is to present Python-based scientific applications and libraries, to discuss general topics regarding the use of Python (such as language design and performance issues), and to share experience using Python in scientific computing education. Paper submission deadline is September 19--so it's not too late to get your contribution in. Andreas

9 years, 4 months

Re: [PyOpenCL] Feature proposal: GPUArray views

by Andreas Kloeckner

On Fri, 9 Sep 2011 13:17:36 +1000, Bogdan Opanchuk <mantihor(a)gmail.com> wrote: > Hello, > > I recently found myself missing a certain feature in PyCuda (and, in > fact, the same can be said about PyOpenCL). Namely, the feature is a > view of a part of GPUArray. > For example, if I have some large array and want to perform some > operation on the part of it, I have to introduce additional parameters > like 'offset' and 'length': > > a = GPUArray(full_size, dtype) > func(a, offset=1024, length=2048) > > In my opinion, this can be too error-prone, and this would be better: > > a = GPUArray(full_size, dtype) > a_view = a.get_view(offset=1024, length=2048) > func(a_view) > > Now func() does not have to worry about offsets and can just work with > a_view as if it were a common GPUArray. This especially helps if > func() is some library function which you cannot/do not want to mess > with. Other advantage of views would be that you can reuse some large > temporary array by creating different views to use at different steps > (instead of creating many temporary arrays which take more memory). > > So, at this point I am just interested whether I am the only one who > will find this feature useful. Unfortunately, in the nearest future I > will not have time to add it myself. I am not even sure whether it > should go to PyCuda/OpenCL or straight to compyte (or is it there > already?). PyCUDA already supports this--just do array[start:end]. PyOpenCL not yet, but I'm meaning to add this at some point, unless compyte gets there first. Andreas

9 years, 4 months

Feature proposal: GPUArray views

by Bogdan Opanchuk

Hello, I recently found myself missing a certain feature in PyCuda (and, in fact, the same can be said about PyOpenCL). Namely, the feature is a view of a part of GPUArray. For example, if I have some large array and want to perform some operation on the part of it, I have to introduce additional parameters like 'offset' and 'length': a = GPUArray(full_size, dtype) func(a, offset=1024, length=2048) In my opinion, this can be too error-prone, and this would be better: a = GPUArray(full_size, dtype) a_view = a.get_view(offset=1024, length=2048) func(a_view) Now func() does not have to worry about offsets and can just work with a_view as if it were a common GPUArray. This especially helps if func() is some library function which you cannot/do not want to mess with. Other advantage of views would be that you can reuse some large temporary array by creating different views to use at different steps (instead of creating many temporary arrays which take more memory). So, at this point I am just interested whether I am the only one who will find this feature useful. Unfortunately, in the nearest future I will not have time to add it myself. I am not even sure whether it should go to PyCuda/OpenCL or straight to compyte (or is it there already?). Best regards, Bogdan

9 years, 4 months

Intel sdk verbosity

by Vincent Favre-Nicolin

Hi, I've been using PyCUDA a lot (see http://pynx.sf.net) and I am just switching to PyOpenCL, which seems to work very nicely, including with CPUs. Right now I am working under Ubuntu with amd+intel+nvidia sdk's, which seem to work nicely along each other and PyOpenCL. Only slightly annoying thing, the intel compiler seems to be a bit verbose when it does not need to be, e.g. I get messages like: ########## Build on <pyopencl.Device 'Intel(R) Core(TM)2 Quad CPU Q9550 @ 2.83GHz' at 0x1af72c0> succeeded, but said: Build started Kernel <Fhkl> was successfully vectorized Done. ########## This is a pretty minor issue, but does someone know how to turn off this message from the intel compiler (while still keeping real warnings, I don't want to completely turn off PyOpenCL reporting of messages). I tried looking in the intel manual, but did not find a compiler option for that. Incidentally, the intel compiler seems much more efficient (3.5x speed) than AMD's for my Intel Core 2 Q9550. Not sure why, either better vectorization or faster sin/cos functions. regards, -- Vincent Favre-Nicolin http://inac.cea.fr CEA/Grenoble Institut Nanosciences & Cryogénie Laboratoire SP2M/Nano-structures et Rayonnement Synchrotron 17, rue des Martyrs 38054 Grenoble Cedex 9 - France Université Joseph Fourier http://www.ujf-grenoble.fr tél: (+33) 4 38 78 95 40 fax: (+33) 4 38 78 51 38

9 years, 4 months

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

PyOpenCL September 2011