PyOpenCL February 2013

pyopencl@tiker.net

9 participants
16 discussions

Segmentation fault in pyopencl.image_from_array

by Jerome Kieffer

Dear Python/OpenCL community, I am pretty new (py)opencl and encountered a problem, maybe it a lack of understanding of openCL, but I found strange python seg-faults: test program: #!/usr/bin/python import numpy, pyopencl ctx = pyopencl.create_some_context() data=numpy.random.random((1024,1024)).astype(numpy.float32) img = pyopencl.image_from_array(ctx, ary=data, mode="r", norm_int=False, num_channels=1) print img System: debian sid: pyopencl2012.1 (the same code works with debian stable and v2011.2) Here is the backtrace obtained with GDB: 0x0000000000000000 in ?? () (gdb) bt #0 0x0000000000000000 in ?? () #1 0x00007ffff340c253 in pyopencl::create_image_from_desc(pyopencl::context const&, unsigned long, _cl_image_format const&, _cl_image_desc&, boost::python::api::object) () from /usr/lib/python2.7/dist-packages/pyopencl/_cl.so #2 0x00007ffff342de36 in _object* boost::python::detail::invoke<boost::python::detail::install_holder<pyopencl::image*>, pyopencl::image* (*)(pyopencl::context const&, unsigned long, _cl_image_format const&, _cl_image_desc&, boost::python::api::object), boost::python::arg_from_python<pyopencl::context const&>, boost::python::arg_from_python<unsigned long>, boost::python::arg_from_python<_cl_image_format const&>, boost::python::arg_from_python<_cl_image_desc&>, boost::python::arg_from_python<boost::python::api::object> >(boost::python::detail::invoke_tag_<false, false>, boost::python::detail::install_holder<pyopencl::image*> const&, pyopencl::image* (*&)(pyopencl::context const&, unsigned long, _cl_image_format const&, _cl_image_desc&, boost::python::api::object), boost::python::arg_from_python<pyopencl::context const&>&, boost::python::arg_from_python<unsigned long>&, boost::python::arg_from_python<_cl_image_format const&>&, boost::python::arg_from_python<_cl_image_desc&>&, boost::python::arg_from_python<boost::python::api::object>&) () from /usr/lib/python2.7/dist-packages/pyopencl/_cl.so #3 0x00007ffff342e06f in boost::python::detail::caller_arity<5u>::impl<pyopencl::image* (*)(pyopencl::context const&, unsigned long, _cl_image_format const&, _cl_image_desc&, boost::python::api::object), boost::python::detail::constructor_policy<boost::python::default_call_policies>, boost::mpl::vector6<pyopencl::image*, pyopencl::context const&, unsigned long, _cl_image_format const&, _cl_image_desc&, boost::python::api::object> >::operator()(_object*, _object*) () from /usr/lib/python2.7/dist-packages/pyopencl/_cl.so #4 0x00007ffff311715b in boost::python::objects::function::call(_object*, _object*) const () from /usr/lib/libboost_python-py27.so.1.49.0 #5 0x00007ffff3117378 in ?? () from /usr/lib/libboost_python-py27.so.1.49.0 #6 0x00007ffff3120593 in boost::python::detail::exception_handler::operator()(boost::function0<void> const&) const () from /usr/lib/libboost_python-py27.so.1.49.0 #7 0x00007ffff3445983 in boost::detail::function::function_obj_invoker2<boost::_bi::bind_t<bool, boost::python::detail::translate_exception<pyopencl::error, void (*)(pyopencl::error const&)>, boost::_bi::list3<boost::arg<1>, boost::arg<2>, boost::_bi::value<void (*)(pyopencl::error const&)> > >, bool, boost::python::detail::exception_handler const&, boost::function0<void> const&>::invoke(boost::detail::function::function_buffer&, boost::python::detail::exception_handler const&, boost::function0<void> const&) () from /usr/lib/python2.7/dist-packages/pyopencl/_cl.so #8 0x00007ffff3120373 in boost::python::handle_exception_impl(boost::function0<void>) () from /usr/lib/libboost_python-py27.so.1.49.0 #9 0x00007ffff3115635 in ?? () from /usr/lib/libboost_python-py27.so.1.49.0 Thanks for your help. If you are not able to reproduce this bug, I should mention it to debian. Cheers, -- Jérôme Kieffer Data analysis unit - ESRF

6 years, 7 months

create Event from cl_event

by Gregor Thalhammer

Dear Andreas, I am currently working on a cython based wrapper for the OpenCL FFT library from AMD: https://github.com/geggo/gpyfft For this I need to create a pyopencl Event instance from a cl_event returned by the library. I attached a patch against recent pyopencl that adds this possibility, similar to the from_cl_mem_as_int() method of the MemoryObject class. Could you please add this to pyopencl. Thanks for your help Gregor

7 years, 2 months

New version and prefix scan problems

by Tomasz Rybak

Sorry if there are two copies of this message. I have sent it to the list but received no confirmation (nor any error) and checked that archive does not show any message from January. I can see that there is already new version (2013.1) in docs, marked "in development". I would like for it not to be released before fixing problems with parallel prefix scan. Problems with scan are only visible on APU Loveland. They do not occur on ION, nor on GTX 460. I do not have access to machine with NVIDIA CC 3.x so I cannot test prefix scan there. I first encountered it in August, and mentioned them in email to the list from 2012-08-08 ("Python3 test failures"). Only recently I had some time and eagerness to look closer into them. Tests still fail on recent git version c31944d1e81a. Failing tests are now in test_algorithm.py, in third group (marked scan-related, starting in line 418). I'll describe my observations of test_scan function. My APU has 2 Computing Units. GenericScanKernel chooses k_group_size to be 4096, max_scan_wg_size to be 256, and max_intervals to 6. The first error occurs when there is enough work to fill two Computing Units - in my case 2**12+5. It looks like there is problem with passing partial result from computations occurring on fist CU to the second one. Prefix sum is computed correctly on the second half of the array but starting with the wrong point. I have printed interval_results array and I have observed that error (difference between the correct value of the interval's first element and actual one) is not the value of any of the elements of interval_results, nor it is difference between interval_results elements. On the other hand difference between real and wanted value is similar (i.e. in the same range) to the difference between interval_results[4] and interval_results[3]. In the test I have run just now the error is 10724571 and the difference is 10719275; I am not sure if this is relevant though. Errors are not repeatable - sometimes they occur for small arrays (e.g. for 2**12+5) sometimes for larger ones (test I have run right now failed for ExclusiveScan of size 2**24+5). The tests' failures also depend on order of tests - after changing order of elements of array scan_test_counts I got failures for different sizes, but always for sizes larger than 2**12. It might be some race condition, but I do not understand new scan fully and cannot point my finger at one place. If there is any additional test I can perform please let me know. I'll try to investigate it further but I am not sure whether it'll work. Best regards. -- Tomasz Rybak GPG/PGP key ID: 2AD5 9860 Fingerprint A481 824E 7DD3 9C0E C40A 488E C654 FB33 2AD5 9860 http://member.acm.org/~tomaszrybak

7 years, 5 months

Re: [PyOpenCL] Abort on MacOSX 10.6

by Conversy Stéphane

Hello everyone, I still have the same problems, I could not fix them. I followed the instructions on the wiki about installing on mac... Maybe this error : > examples/demo_array.py gives: > File "/Library/Frameworks/Python.framework/Versions/7.3/lib/python2.7/site-packages/pyopencl-2013.1-py2.7-macosx-10.5-i386.egg/pyopencl/compyte/dtypes.py", line 173, in dtype_to_ctype > raise ValueError, "unable to map dtype '%s'" % dtype > ValueError: unable to map dtype 'int32' is a symptom of a typical mistake that someone hopefully recognizes ? Best regards, Stéphane > Hello, > > I tried to compile pyopencl on my Mac (10.6), using the python distribution from Enthough (2.7.3). > Some of the demos work, but: > > examples/demo_array.py gives: > File "/Library/Frameworks/Python.framework/Versions/7.3/lib/python2.7/site-packages/pyopencl-2013.1-py2.7-macosx-10.5-i386.egg/pyopencl/compyte/dtypes.py", line 173, in dtype_to_ctype > raise ValueError, "unable to map dtype '%s'" % dtype > ValueError: unable to map dtype 'int32' > > examples/demo_mandelbrot.py gives: > Abort trap > > with the stack given at the end of the msg. > > Any tips ? > > Best regards, > > Stéphane > > Thread 0 Crashed: Dispatch queue: com.apple.main-thread > 0 libSystem.B.dylib 0x977a9c5a __kill + 10 > 1 libSystem.B.dylib 0x977a9c4c kill$UNIX2003 + 32 > 2 libSystem.B.dylib 0x9783c5a5 raise + 26 > 3 libSystem.B.dylib 0x97852679 __abort + 124 > 4 libSystem.B.dylib 0x97835334 release_file_streams_for_task + 0 > 5 ...pple.ATIRadeonX3000GLDriver 0x190a7c77 glrCompCreateKernel + 7015 > 6 ...pple.ATIRadeonX3000GLDriver 0x190ab2a4 glrCompLoadBinary + 2852 > 7 ...pple.ATIRadeonX3000GLDriver 0x190ab3ff glrCompBuildProgram + 287 > 8 com.apple.opencl 0x973674f6 clBuildProgram + 2643 > 9 _cl.so 0x011f9ecf pyopencl::program::build(std::string, pyopenclboost::python::api::object) + 879 > 10 _cl.so 0x011e8452 pyopenclboost::python::objects::caller_py_function_impl<pyopenclboost::python::detail::caller<void (pyopencl::program::*)(std::string, pyopenclboost::python::api::object), pyopenclboost::python::default_call_policies, pyopenclboost::mpl::vector4<void, pyopencl::program&, std::string, pyopenclboost::python::api::object> > >::operator()(_object*, _object*) + 322 > 11 _cl.so 0x01241d02 pyopenclboost::python::objects::function::call(_object*, _object*) const + 322 > 12 _cl.so 0x01241fb7 pyopenclboost::detail::function::void_function_ref_invoker0<pyopenclboost::python::objects::(anonymous namespace)::bind_return, void>::invoke(pyopenclboost::detail::function::function_buffer&) + 39 > 13 _cl.so 0x01250177 pyopenclboost::python::detail::exception_handler::operator()(pyopenclboost::function0<void> const&) const + 279 > 14 _cl.so 0x012266c4 pyopenclboost::detail::function::function_obj_invoker2<pyopenclboost::_bi::bind_t<bool, pyopenclboost::python::detail::translate_exception<pyopencl::error, void (*)(pyopencl::error const&)>, pyopenclboost::_bi::list3<pyopenclboost::arg<1>, pyopenclboost::arg<2>, pyopenclboost::_bi::value<void (*)(pyopencl::error const&)> > >, bool, pyopenclboost::python::detail::exception_handler const&, pyopenclboost::function0<void> const&>::invoke(pyopenclboost::detail::function::function_buffer&, pyopenclboost::python::detail::exception_handler const&, pyopenclboost::function0<void> const&) + 36 > 15 _cl.so 0x0124fd29 pyopenclboost::python::handle_exception_impl(pyopenclboost::function0<void>) + 153 > 16 _cl.so 0x0123ef7e function_call + 78 > 17 org.python.python 0x0000f635 PyObject_Call + 85 > 18 org.python.python 0x000c9616 PyEval_EvalFrameEx + 14294 > 19 org.python.python 0x000cd02a PyEval_EvalCodeEx + 2042 > 20 org.python.python 0x000cb253 PyEval_EvalFrameEx + 21523 > 21 org.python.python 0x000cd02a PyEval_EvalCodeEx + 2042 > 22 org.python.python 0x000cb253 PyEval_EvalFrameEx + 21523 > 23 org.python.python 0x000cd02a PyEval_EvalCodeEx + 2042 > 24 org.python.python 0x000cb253 PyEval_EvalFrameEx + 21523 > 25 org.python.python 0x000cd02a PyEval_EvalCodeEx + 2042 > 26 org.python.python 0x000cb253 PyEval_EvalFrameEx + 21523 > 27 org.python.python 0x000cc053 PyEval_EvalFrameEx + 25107 > 28 org.python.python 0x000cd02a PyEval_EvalCodeEx + 2042 > 29 org.python.python 0x000cb253 PyEval_EvalFrameEx + 21523 > 30 org.python.python 0x000cc053 PyEval_EvalFrameEx + 25107 > 31 org.python.python 0x000cd02a PyEval_EvalCodeEx + 2042 > 32 org.python.python 0x00042282 function_call + 162 > 33 org.python.python 0x0000f635 PyObject_Call + 85 > 34 org.python.python 0x00021fe6 instancemethod_call + 422 > 35 org.python.python 0x0000f635 PyObject_Call + 85 > 36 org.python.python 0x0007cff7 slot_tp_init + 87 > 37 org.python.python 0x0007ba20 type_call + 176 > 38 org.python.python 0x0000f635 PyObject_Call + 85 > 39 org.python.python 0x000c9616 PyEval_EvalFrameEx + 14294 > 40 org.python.python 0x000cd02a PyEval_EvalCodeEx + 2042 > 41 org.python.python 0x000cd1b7 PyEval_EvalCode + 87 > 42 org.python.python 0x000f1898 PyRun_FileExFlags + 168 > 43 org.python.python 0x000f27c3 PyRun_SimpleFileExFlags + 867 > 44 org.python.python 0x0010ba58 Py_Main + 3560 > 45 org.python.python 0x00001fb6 0x1000 + 4022 > Stéphane Conversy Enseignant Chercheur en IHM - ENAC http://lii-enac.fr/~conversy +33 5 62 17 40 19

7 years, 5 months

Re: [PyOpenCL] pyopencl.RuntimeError: Context failed: device not available

by Andreas Kloeckner

Hi Antonio, Antonio Rieser <tonyrieser(a)gmail.com> writes: > I just built and installed pyopencl from source on an old Dell > Inspiron 1440 running Fedora 18 with the Intel SDK OpenCL > implementation and Python version 3.3. I believe the installation went > ok (after a few hiccups), but I now get a "device not available" > runtime error when I try to run benchmark.py (as well as any other > example program that does not give me a syntax error), and I don't see > how to resolve it. The output of "python3.3 benchmark.py" is: > > Execution time of test without OpenCL: 0.17273521423339844 s > =============================================================== > Platform name: Intel(R) OpenCL > Platform profile: FULL_PROFILE > Platform vendor: Intel(R) Corporation > Platform version: OpenCL 1.2 LINUX > --------------------------------------------------------------- > Device name: Intel(R) Core(TM)2 Duo CPU T6600 @ 2.20GHz > Device type: CPU > Device memory: 2974 MB > Device max clock speed: 2200 MHz > Device compute units: 2 > Device max work group size: 1024 > Device max work item sizes: [1024, 1024, 1024] > Traceback (most recent call last): > File "benchmark.py", line 46, in <module> > ctx = cl.Context([device]) > pyopencl.RuntimeError: Context failed: device not available Is your CPU new enough to be supported by the Intel CL SDK? If not, try the AMD one--this is a bit more compatible with old CPUs. HTH, Andreas

7 years, 7 months

git repositories on git.tiker.net

by Tomasz Rybak

Is everything OK with git repositories on git.tiker.net? When I tried to pull changes I got $ git pull From http://git.tiker.net/trees/pycuda + 2e20f2c...00cf720 master -> origin/master (forced update) Already up-to-date. Cloning PyCUDA repository gives last change from 2011-12-07: commit 00cf72000946edc7b6f684748be9a888e3aef20e Author: Andreas Kloeckner <inform(a)tiker.net> Date: Wed Dec 7 10:23:20 2011 -0500 Bump version (for CUDA 3.1 fix), release. Status on pyopencl gives: $ git status # On branch master # Your branch is ahead of 'origin/master' by 544 commits. # nothing to commit (working directory clean) At the same time repositories on github have changes from Feb 2013. What's going on? As I have git.tiker.net mentioned as master repository in my packages, should I change it to github? Best regards. -- Tomasz Rybak <tomasz.rybak(a)post.pl> GPG/PGP key ID: 2AD5 9860 Fingerprint A481 824E 7DD3 9C0E C40A 488E C654 FB33 2AD5 9860 http://member.acm.org/~tomaszrybak

7 years, 7 months

pyopencl.RuntimeError: Context failed: device not available

by Antonio Rieser

Hello! I just built and installed pyopencl from source on an old Dell Inspiron 1440 running Fedora 18 with the Intel SDK OpenCL implementation and Python version 3.3. I believe the installation went ok (after a few hiccups), but I now get a "device not available" runtime error when I try to run benchmark.py (as well as any other example program that does not give me a syntax error), and I don't see how to resolve it. The output of "python3.3 benchmark.py" is: Execution time of test without OpenCL: 0.17273521423339844 s =============================================================== Platform name: Intel(R) OpenCL Platform profile: FULL_PROFILE Platform vendor: Intel(R) Corporation Platform version: OpenCL 1.2 LINUX --------------------------------------------------------------- Device name: Intel(R) Core(TM)2 Duo CPU T6600 @ 2.20GHz Device type: CPU Device memory: 2974 MB Device max clock speed: 2200 MHz Device compute units: 2 Device max work group size: 1024 Device max work item sizes: [1024, 1024, 1024] Traceback (most recent call last): File "benchmark.py", line 46, in <module> ctx = cl.Context([device]) pyopencl.RuntimeError: Context failed: device not available Any help is heartily appreciated. Best regards, Antonio

7 years, 7 months

Performance results

by Antonio Beamud Montero

Hi all again: I've created a test to measure performance... You can test it, the code is here: http://pastebin.com/Nye5Axm8 I'm using two arrays and the results, only changing the size of the first only in two elements, give me a lot of performance loss in the gpu only. Time for ASIZE: 29120 [GPU]: 0.296602 s Time for ASIZE: 29120 [CPU]: 3.12564 s Time for ASIZE: 29122 [GPU]: 11.2411 s Time for ASIZE: 29122 [CPU]: 3.13552 s Why this difference in the gpu performance? I'm using pyopencl 0.92 (Ubuntu 12.04 version), and my graphic card is a Radeon HD6450. A lot of thanks.

7 years, 7 months

Using different types in buffers

by Antonio Beamud Montero

Hi: I was testing using a result array of booleans, the problem is accessing by index to the correct element... What's the correct way of doing this? For example, with this buffers declarations: a = numpy.random.randint(-100000000, 100000000, 10).astype(numpy.int32) r = numpy.zeros(10, numpy.bool) a_buf = cl.Buffer(ctx, mf.READ_ONLY | mf.COPY_HOST_PTR, hostbuf=a) dest_buf = cl.Buffer(ctx, mf.WRITE_ONLY | mf.COPY_HOST_PTR, hostbuf=r) and this kernel: __kernel void test(__global const int *a, __global bool *c ) { int agid = get_global_id(0); if (agid == 1) c[agid] = true; } The c buffer contains the next values (initialized to false 10 elements): [False False False False True False False False False False] i.e. is using 1 * 4 (int size, 4 bytes) as index value... I'm a little confused :-)

7 years, 7 months

Re: [PyOpenCL] Handling Multiple kernels

by Calle Snickare

Hello, Is my question too hard or too trivial? I can't find any examples or info on the web on this. I'll try to reformulate: I want to run two different kernel functions in succession, with the same variables/input. They are both written in the same c-file. This is what I run in the host-code: kernel_1 = prg.function_1 kernelObj_1= kernel_1(queue, globalSize, localSize, ins.data, ranluxcltab) kernelObj_1.wait() kernel_2 = prg.function_2 kernelObj_2 = kernel_2(queue, globalSize, localSize, ins.data, ranluxcltab) kernelObj_2.wait() Is this correct? If so - I'm running out of memory faster than I expect. Is the same data really being used in this way, or is it duplicated? Cheers, Calle On Thu, Jan 31, 2013 at 9:11 AM, Calle Snickare <problembarnet(a)gmail.com>wrote: > Hello again, > I checked again and I need to reduce my number of threads run by a factor > 4 not to get "out of memory" error. This seems very strange since the idea > is that I want to use the same memory for seeds etc when running the > initialize kernel as running my main kernel. Is there something wrong in my > kernel invocation? > > Cheers, Calle > > > On Mon, Jan 28, 2013 at 3:40 PM, Calle Snickare <problembarnet(a)gmail.com>wrote: > >> Hello, >> I am currently trying to implement Ranlux in one of my programs. My >> kernel will be re-run several times with the same seeds, so I don't want to >> include the Ranlux initialization in it as I only want to do this once >> (right?). I also want to make sure to use the same memory between the runs. >> So I figure that I solve this by having two kernels: one kernel that >> initializes Ranlux (run this once at the beginning), as well as my "main" >> kernel. They will both be written in the same c-file. >> >> Here is some of the code. At first I had some strange errors getting it >> to work. Now I can get it to run, but it feels like it runs out of memory >> quicker than it should. Am I approaching this the wrong way? >> >> >> Host code: >> ctx = cl.create_some_context() >> queueProperties = cl.command_queue_properties.PROFILING_ENABLE >> queue = cl.CommandQueue(ctx, properties=queueProperties) >> >> mf = cl.mem_flags >> dummyBuffer = np.zeros(nbrOfThreads * 28, dtype=np.uint32) >> ins = cl.array.to_device(queue, (np.random.randint(0, high = 2 ** 31 - 1, >> size = (nbrOfThreads))).astype(np.uint32)) >> ranluxcltab = cl.Buffer(ctx, mf.READ_WRITE, size=0, hostbuf=dummyBuffer) >> >> kernelCode_r = open(os.path.dirname(__file__) + 'ranlux_test_kernel.c', >> 'r').read() >> kernelCode = kernelCode_r % replacements >> >> prg = (cl.Program(ctx, kernelCode).build(options=programBuildOptions)) >> >> kernel_init = prg.ranlux_init_kernel >> kernelObj_init = kernel_init(queue, globalSize, localSize, ins.data, >> ranluxcltab) >> >> kernelObj_init.wait() >> >> kernel = prg.ranlux_test_kernel >> kernelObj = kernel(queue, globalSize, localSize, ins.data, ranluxcltab) >> kernelObj.wait() >> >> Kernel Code: >> #pragma OPENCL EXTENSION cl_khr_fp64 : enable >> #define RANLUXCL_SUPPORT_DOUBLE >> #include "pyopencl-ranluxcl.cl" // Ranlux source-code >> #define RANLUXCL_LUX 4 >> >> __kernel void ranlux_init_kernel(__global uint *ins, __global >> ranluxcl_state_t *ranluxcltab) >> { >> //ranluxclstate stores the state of the generator. >> ranluxcl_state_t ranluxclstate; >> >> ranluxcl_initialization(ins, ranluxcltab); >> } >> >> __kernel void ranlux_test_kernel(__global uint *ins, __global >> ranluxcl_state_t *ranluxcltab) >> { >> uint threadId = get_global_id(0) + get_global_id(1) * >> get_global_size(0); >> >> //ranluxclstate stores the state of the generator. >> ranluxcl_state_t ranluxclstate; >> >> //Download state into ranluxclstate struct. >> ranluxcl_download_seed(&ranluxclstate, ranluxcltab); >> >> double randomnr; >> randomnr = ranluxcl64(&ranluxclstate); >> /* DO STUFF */ >> >> >> //Upload state again so that we don't get the same >> //numbers over again the next time we use ranluxcl. >> ranluxcl_upload_seed(&ranluxclstate, ranluxcltab); >> } >> >> >> Cheers, >> Calle >> > >

7 years, 7 months

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

PyOpenCL February 2013