PyOpenCL December 2015

pyopencl@tiker.net

8 participants
10 discussions

by Andreas Kloeckner

Zac Diggum <Diggum(a)gmx.de> writes: > You made it! > Today I saw v2015.2.4 on Christoph Gohlke's website. Great work! Thank > you Andreas, Christoph, and Santa. You're welcome! Thanks for providing the initial impulse to get this done. > One last question: is it possible to build the new version for Python > 2.7 as well? If not, I might just have found my final reason to switch > to 3.5. >From PyOpenCL's perspective, there shouldn't be an obstacle to building for 2.7, but there may be something Windows-specific lurking that I'm unaware of. At any rate--I'm enjoying using 3.5 for my day-to-day work. It's better in a lot of little ways than Py2. > merry Christmas / fröhliche Weihnachten... To you too! Andreas

5 years, 1 month

What's the point of ENABLE_GL as an option?

by Keith Brafford

Is there a good reason why ENABLE_GL is an option? If I have PyOpenCL compiled with OpenGL support but I choose not to use it, isn't that the same thing as having ENABLE_GL as False? --Keith

5 years, 1 month

Build error on Win 7 x64 using Microsoft Visual C++ Compiler for Python

by Prashant Saxena

Hi, pyopencl-master(Downloaded from github)Win7 x64Microsoft Visual C++ Compiler for PythonPython 2.7.2Nvidia Cuda 7.5 Commands:python configure.pysiteconf.py:CL_TRACE = False CL_ENABLE_GL = True CL_INC_DIR = ["C:\\Program Files\\NVIDIA GPU Computing Toolkit\\CUDA\\v7.5\\include"] CL_LIB_DIR = ["C:\\Program Files\\NVIDIA GPU Computing Toolkit\\CUDA\\v7.5\\lib\\x64"] CL_LIBNAME = ['OpenCL'] CXXFLAGS = ['-std=c++0x'] LDFLAGS = []python.exe setup.py build_ext --compiler=msvc Errors:generating cffi module 'build\\temp.win-amd64-2.7\\Release\\pyopencl._cffi.cpp' building 'pyopencl._cffi' extension C:\Users\john\AppData\Local\Programs\Common\Microsoft\Visual C++ for Python\9.0\ VC\Bin\amd64\cl.exe /c /nologo /Ox /MD /W3 /GS- /DNDEBUG -DPYGPU_PACKAGE=pyopenc l -DPYGPU_PYOPENCL=1 -DHAVE_GL=1 "-IC:\Program Files\NVIDIA GPU Computing Toolki t\CUDA\v7.5\include" -Isrc/c_wrapper/ -IC:\Python27\include -IC:\Python27\PC /Tp build\temp.win-amd64-2.7\Release\pyopencl._cffi.cpp /Fobuild\temp.win-amd64-2.7\ Release\build\temp.win-amd64-2.7\Release\pyopencl._cffi.obj -std=c++0x cl : Command line warning D9002 : ignoring unknown option '-std=c++0x' pyopencl._cffi.cpp C:\Users\john\AppData\Local\Programs\Common\Microsoft\Visual C++ for Python\9.0\ VC\Bin\amd64\cl.exe /c /nologo /Ox /MD /W3 /GS- /DNDEBUG -DPYGPU_PACKAGE=pyopenc l -DPYGPU_PYOPENCL=1 -DHAVE_GL=1 "-IC:\Program Files\NVIDIA GPU Computing Toolki t\CUDA\v7.5\include" -Isrc/c_wrapper/ -IC:\Python27\include -IC:\Python27\PC /Tp src/c_wrapper/wrap_cl.cpp /Fobuild\temp.win-amd64-2.7\Release\src/c_wrapper/wrap _cl.obj -std=c++0x cl : Command line warning D9002 : ignoring unknown option '-std=c++0x' wrap_cl.cpp d:\pyopencl-master\src\c_wrapper\wrap_cl_core.h(53) : error C2061: syntax error : identifier 'uint32_t' d:\pyopencl-master\src\c_wrapper\wrap_cl_core.h(59) : error C2065: 'add' : undec lared identifier d:\pyopencl-master\src\c_wrapper\wrap_cl_core.h(59) : error C2143: syntax error : missing ')' before 'const' d:\pyopencl-master\src\c_wrapper\wrap_cl_core.h(59) : error C2059: syntax error : ')' d:\pyopencl-master\src\c_wrapper\wrap_cl_core.h(64) : error C2061: syntax error : identifier 'uint32_t' d:\pyopencl-master\src\c_wrapper\wrap_cl_core.h(66) : error C2061: syntax error : identifier 'uint32_t' d:\pyopencl-master\src\c_wrapper\wrap_cl_core.h(70) : error C2061: syntax error : identifier 'uint32_t' d:\pyopencl-master\src\c_wrapper\wrap_cl_core.h(101) : error C2061: syntax error : identifier 'uint32_t' It would be great if we'll be able to build pyopencl using "Microsoft Visual C++ Compiler for Python". CheersPrashant

5 years, 1 month

Import error on Ubuntu 14.04

by Prashant Saxena

Hi,pyopencl-masterUbuntu x86_64 14.04Nvidia GTX 750 TiGCC 4.8.4 siteconf.pyCL_TRACE = False CL_ENABLE_GL = False CL_INC_DIR = ['usr/local/cuda-7.5/include'] CL_LIB_DIR = ['usr/local/cuda-7.5/lib64'] CL_LIBNAME = ['OpenCL'] CXXFLAGS = ['-std=c++0x'] LDFLAGS = [] After building and installing I'm getting this error: import pyopencl as cl File "/usr/local/lib/python2.7/dist-packages/pyopencl-2015.2.3-py2.7-linux-x86_64.egg/pyopencl/__init__.py", line 35, in <module> import pyopencl.cffi_cl as _cl File "/usr/local/lib/python2.7/dist-packages/pyopencl-2015.2.3-py2.7-linux-x86_64.egg/pyopencl/cffi_cl.py", line 37, in <module> from .compyte.array import f_contiguous_strides, c_contiguous_strides ImportError: No module named compyte.array Secondly If I switch on:CL_ENABLE_GL = TrueHow and where do I inlude 'GL' library? CheersPrashant

5 years, 1 month

Re: [PyOpenCL] PyOpenCL on Windows

by Andreas Kloeckner

Christoph Gohlke <cgohlke(a)uci.edu> writes: > <lists(a)weasel.tiker.net> bounced. Resending to <andreask(a)illinois.edu>. > > > Hi Andreas, > > sorry for the delay. > > I tried building PyOpenCL 2015.2.x on Windows and noticed several issues: > > > 1. The PyPI source distribution is missing src/c_wrapper/mingw-std-threads. > > Using git master for now. Fixed. [ea3d7a6] > 2. Setup.py fails if git not found. > > Using this workaround: > > diff --git a/aksetup_helper.py b/aksetup_helper.py > index b7378b5..29ed40a 100644 > --- a/aksetup_helper.py > +++ b/aksetup_helper.py > @@ -703,6 +703,7 @@ def substitute(substitutions, fname): > def _run_git_command(cmd): > git_error = None > from subprocess import Popen, PIPE > + stdout = None > try: > popen = Popen(["git"] + cmd, stdout=PIPE) > stdout, stderr = popen.communicate() > @@ -722,8 +723,11 @@ def _run_git_command(cmd): > print(git_error) > print("Hit Ctrl-C now if you'd like to think about the > situation.") > print(DASH_SEPARATOR) > - count_down_delay(delay=5) > - return stdout.decode("utf-8"), git_error > + count_down_delay(delay=0) > + if stdout: > + return stdout.decode("utf-8"), git_error > + else: > + return '', None > > > def check_git_submodules(): Merged, thanks. [e6ee3ca] > 3. PyOpenCL 2015.2.x was apparently rewritten specifically for gcc > compilers. The default platform compilers for Python on Windows, Visual > Studio 2008, 2010, and 2015 no longer compile the code. Same for Intel > Compilers 2013 and 2016. Too many different errors to report. It uses a healthy dose of C++11. Clang and g++ are fine, FWIW, but I can see why MSVC and Intel are unhappy. :/ I don't know how ready MS's (recent) Clang frontend for their compilers is, but that may be a viable solution. In the meantime, I've mentioned mingwpy in the README [5f08e13]. > The workaround is to use mingwpy compilers. However, they do not support > Python 3.5, I can not debug the builds, and cffi might have issues: > > pyopencl._cffi.cpp:365:68: warning: unknown conversion type character > 'z' in format This is spurious. The code causing this looks like this: _CFFI_UNUSED_FN static PyObject **_cffi_unpack_args(PyObject *args_tuple, Py_ssize_t expected, const char *fnname) { if (PyTuple_GET_SIZE(args_tuple) != expected) { PyErr_Format(PyExc_TypeError, "%.150s() takes exactly %zd arguments (%zd given)", fnname, expected, PyTuple_GET_SIZE(args_tuple)); return NULL; } return &PyTuple_GET_ITEM(args_tuple, 0); /* pointer to the first item, the others follow */ } and %zd is allowed in PyErr_Format: https://docs.python.org/2/c-api/string.html#c.PyString_FromFormat > 4. strdup failed to link: > > wrap_cl.cpp:(.text+0x42f): undefined reference to `strdup' > > Using _strdup instead: > > diff --git a/src/c_wrapper/wrap_cl.h b/src/c_wrapper/wrap_cl.h > index 41e57c2..576de6c 100644 > --- a/src/c_wrapper/wrap_cl.h > +++ b/src/c_wrapper/wrap_cl.h > @@ -31,6 +31,7 @@ > #if defined(_WIN32) > #define NOMINMAX > #include <windows.h> > +#define strdup _strdup > #else > #include <unistd.h> > #endif Merged, thanks. [971ad78] > 5. CL_KERNEL_ARG_TYPE_const' was not declared in this scope: > > src/c_wrapper/wrap_constants.cpp:8:24: error: 'CL_KERNEL_ARG_TYPE_const' > was not declared in this scope > > Disabling CL_KERNEL_ARG_TYPE_const for now: > > diff --git a/src/c_wrapper/wrap_constants.cpp > b/src/c_wrapper/wrap_constants.cpp > index 0685b1f..661d7e8 100644 > --- a/src/c_wrapper/wrap_constants.cpp > +++ b/src/c_wrapper/wrap_constants.cpp > @@ -613,7 +614,7 @@ void populate_constants(void(*add)(const char*, > const char*, int64_t value)) > // kernel_arg_type_qualifier > #if PYOPENCL_CL_VERSION >= 0x1020 > ADD_ATTR("kernel_arg_type_qualifier", KERNEL_ARG_TYPE_, NONE); > - ADD_ATTR("kernel_arg_type_qualifier", KERNEL_ARG_TYPE_, CONST); > + //ADD_ATTR("kernel_arg_type_qualifier", KERNEL_ARG_TYPE_, CONST); > ADD_ATTR("kernel_arg_type_qualifier", KERNEL_ARG_TYPE_, RESTRICT); > ADD_ATTR("kernel_arg_type_qualifier", KERNEL_ARG_TYPE_, VOLATILE); > #endif It's weird that that would be missing while the others are there, but so be it. I've made an #ifdef gate to check whether it's there. [7a9da07] > 6. 32 bit build fails with: > > src/c_wrapper/error.h:255:35: error: invalid conversion from 'cl_int > (__attribute__((__stdcall__)) *)(cl_mem) {aka int > (__attribute__((__stdcall__)) *)(_cl_mem*)}' to 'cl_int (*)(_cl_mem*) > {aka int (*)(_cl_mem*)}' [-fpermissive] > call_guarded(func, #func, args) Missing __stdcall in function pointer. Worked around. [d19d94cf] > 7. 64 bit builds succeed and pyopencl can be imported, but running any > test results in immediate segfaults in _cffi.pyd. > > I used latest CUDA and Intel OpenCL SDKs. CFFI is version 1.3.1, > compiled with Visual Studio. I'll try to debug the crash when I find the > time. Hard for me to investigate without a Windows build environment. If I can help somehow, please let me know. I'll probably push out a release of these fixes just to prevent some grief for folks trying to build on Windows. > Greetings to Urbana-Champaign. I worked at Loomis Lab for ~8 years. :) I can see the place from my office in CS right now. Hope you enjoyed it here! Andreas

5 years, 1 month

Re: [PyOpenCL] PyOpenCL on Windows

by Andreas Kloeckner

Ugh, so many issues. Sorry about that, and thanks for taking the time to list them. I'll work on fixing them. CC'ing the list so they're staying informed on what's up. Andreas Christoph Gohlke <cgohlke(a)uci.edu> writes: > <lists(a)weasel.tiker.net> bounced. Resending to <andreask(a)illinois.edu>. > > > Hi Andreas, > > sorry for the delay. > > I tried building PyOpenCL 2015.2.x on Windows and noticed several issues: > > > 1. The PyPI source distribution is missing src/c_wrapper/mingw-std-threads. > > Using git master for now. > > > 2. Setup.py fails if git not found. > > Using this workaround: > > diff --git a/aksetup_helper.py b/aksetup_helper.py > index b7378b5..29ed40a 100644 > --- a/aksetup_helper.py > +++ b/aksetup_helper.py > @@ -703,6 +703,7 @@ def substitute(substitutions, fname): > def _run_git_command(cmd): > git_error = None > from subprocess import Popen, PIPE > + stdout = None > try: > popen = Popen(["git"] + cmd, stdout=PIPE) > stdout, stderr = popen.communicate() > @@ -722,8 +723,11 @@ def _run_git_command(cmd): > print(git_error) > print("Hit Ctrl-C now if you'd like to think about the > situation.") > print(DASH_SEPARATOR) > - count_down_delay(delay=5) > - return stdout.decode("utf-8"), git_error > + count_down_delay(delay=0) > + if stdout: > + return stdout.decode("utf-8"), git_error > + else: > + return '', None > > > def check_git_submodules(): > > > 3. PyOpenCL 2015.2.x was apparently rewritten specifically for gcc > compilers. The default platform compilers for Python on Windows, Visual > Studio 2008, 2010, and 2015 no longer compile the code. Same for Intel > Compilers 2013 and 2016. Too many different errors to report. > > The workaround is to use mingwpy compilers. However, they do not support > Python 3.5, I can not debug the builds, and cffi might have issues: > > pyopencl._cffi.cpp:365:68: warning: unknown conversion type character > 'z' in format > > > 4. strdup failed to link: > > wrap_cl.cpp:(.text+0x42f): undefined reference to `strdup' > > Using _strdup instead: > > diff --git a/src/c_wrapper/wrap_cl.h b/src/c_wrapper/wrap_cl.h > index 41e57c2..576de6c 100644 > --- a/src/c_wrapper/wrap_cl.h > +++ b/src/c_wrapper/wrap_cl.h > @@ -31,6 +31,7 @@ > #if defined(_WIN32) > #define NOMINMAX > #include <windows.h> > +#define strdup _strdup > #else > #include <unistd.h> > #endif > > > 5. CL_KERNEL_ARG_TYPE_const' was not declared in this scope: > > src/c_wrapper/wrap_constants.cpp:8:24: error: 'CL_KERNEL_ARG_TYPE_const' > was not declared in this scope > > Disabling CL_KERNEL_ARG_TYPE_const for now: > > diff --git a/src/c_wrapper/wrap_constants.cpp > b/src/c_wrapper/wrap_constants.cpp > index 0685b1f..661d7e8 100644 > --- a/src/c_wrapper/wrap_constants.cpp > +++ b/src/c_wrapper/wrap_constants.cpp > @@ -613,7 +614,7 @@ void populate_constants(void(*add)(const char*, > const char*, int64_t value)) > // kernel_arg_type_qualifier > #if PYOPENCL_CL_VERSION >= 0x1020 > ADD_ATTR("kernel_arg_type_qualifier", KERNEL_ARG_TYPE_, NONE); > - ADD_ATTR("kernel_arg_type_qualifier", KERNEL_ARG_TYPE_, CONST); > + //ADD_ATTR("kernel_arg_type_qualifier", KERNEL_ARG_TYPE_, CONST); > ADD_ATTR("kernel_arg_type_qualifier", KERNEL_ARG_TYPE_, RESTRICT); > ADD_ATTR("kernel_arg_type_qualifier", KERNEL_ARG_TYPE_, VOLATILE); > #endif > > > 6. 32 bit build fails with: > > src/c_wrapper/error.h:255:35: error: invalid conversion from 'cl_int > (__attribute__((__stdcall__)) *)(cl_mem) {aka int > (__attribute__((__stdcall__)) *)(_cl_mem*)}' to 'cl_int (*)(_cl_mem*) > {aka int (*)(_cl_mem*)}' [-fpermissive] > call_guarded(func, #func, args) > > > 7. 64 bit builds succeed and pyopencl can be imported, but running any > test results in immediate segfaults in _cffi.pyd. > > I used latest CUDA and Intel OpenCL SDKs. CFFI is version 1.3.1, > compiled with Visual Studio. I'll try to debug the crash when I find the > time. > > > Greetings to Urbana-Champaign. I worked at Loomis Lab for ~8 years. > > Christoph > > > > On 12/3/2015 2:53 PM, Andreas Kloeckner wrote: >> Christoph, >> >> Zac Diggum (cc'd) reported that you had some trouble getting PyOpenCl to >> compile on Windows. That makes sense since the 2015.2 series is >> essentially a rewrite of the wrapper. I'd be interested in hearing what >> trouble you ran into, and if I can somehow help getting it to >> work. Could you report what issues you ran into? >> >> Thanks! >> Andreas >> >

5 years, 1 month

Transfer and modify a numpy array on device - example

by Schock, Jonathan

Hello, I searched a lot around for this and couldn't find any answers - so I'm asking here: Can someone please provide me with a working example of allocating a 3D numpy array, moving it to the device, change each xy-2D slice by one work item using the work item's gid and get the result back on the host? There are also some open questions on stackoverflow concerning the same or similar issues, so I feel, I'm not the only one having trouble understanding the basic handling of arrays/numpy/openCL/pyopenCL in a consistent way: http://stackoverflow.com/questions/17928445/why-use-pyopencl-array-array http://stackoverflow.com/questions/33179043/convert-opencl-image-to-pyopenc… and especially: http://stackoverflow.com/questions/32188144/pyopencl-indexing-3d-arrays-ins… Thanks in advance

5 years, 1 month

PyOpenCL on Windows

by Andreas Kloeckner

Christoph, Zac Diggum (cc'd) reported that you had some trouble getting PyOpenCl to compile on Windows. That makes sense since the 2015.2 series is essentially a rewrite of the wrapper. I'd be interested in hearing what trouble you ran into, and if I can somehow help getting it to work. Could you report what issues you ran into? Thanks! Andreas

5 years, 1 month

PyOpenCL 2015.2 Windows

by Zac Diggum

Hi there, today I asked Christoph Gohlke, who runs a website offering precompiled Python packages for Windows OS ( http://www.lfd.uci.edu/~gohlke/pythonlibs/ ), to update to PyOpenCL 2015.2. He told me he wasn't able to compile versions newer than 2015.1 on Windows using Visual Studio 2008, 2010 & 2015, Intel 2013 & 2016, or mingwpy. Any ideas?

5 years, 1 month

Re: [PyOpenCL] OpenGL corruption with OpenCL and OpenGL in separate threads

by Henry Gomersall

Yes, this is plausible, but I'm a little doubtful - it seems rather too repeatable a problem (the same lines seem to get corrupted each time, and not in a spurious way, but in a drop-out way. That said, I'll keep it in mind during my investigations! I'm inclined at the moment to think it could be the beignet drivers. I've found in the past they don't always behave quite right, and it could easily be those. Thanks, Henry On 25/11/15 18:54, CRV§ADER//KY wrote: > I suggest you check your opencl code for buffer overflows. Not sure how > memory segmentation works in VRAM but I would expect two threads of the > same process to be able to freely write to each other's memory, just > like in regular RAM. > > Try replacing your opencl code with something trivial (that simply keeps > the gpu busy) and see if the problem disappears. > > On 25 Nov 2015 5:28 pm, "Karl Czajkowski" <karlcz(a)isi.edu > <mailto:karlcz@isi.edu>> wrote: > > On Nov 25, Henry Gomersall modulated: > > > The OpenGL context is created independently of the OpenCL context and > > exists in a different thread. I'm currently making no attempt to share > > data between OpenGL and OpenCL - all the data passes via the CPU. > > > > How is this sharing via CPU done? Are you copying data into opengl > vertex buffers or textures via opengl API calls? Are these calls done > in the opengl thread at the right point in the rendering loop? > > At the very least, you need to do something to synchronize your reads > of results from the opencl thread and your writes to the opengl > thread. I would expect you'd copy opencl results into Python during > the opencl thread (between kernel runs) and stick them in a > synchronized Python structure to then pull them out and send to opengl > from the opengl thread. > > > > I'm finding that I'm getting substantial artifacts in the OpenGL > > rendering. Specifically, I get a heavy flickering of the rendering > - it > > looks like e.g. certain line segments are not being drawn (it's a > simple > > line drawing program) for some frame. > > > > Is the renderer using line vertices copied over from opencl results? > Do the missing lines make sense as rendering a partially copied array? > > Also, are you using double-buffering in opengl so that only completely > rendered frames will be swapped to the display? Otherwise, you could > also see parts of the scene drawn or missing depending on when they > happen relative to the vertical blanking period and buffer clearing. > > > > Curiously, it doesn't break _every_ run. About 1 run in 10 works > > perfectly, with no visual problems at all. I wonder if there is some > > race condition causing a (setup?) problem. > > > > This could be dumb luck that the workers get phase-locked and > synchronized by accident. > > > Karl > > > _______________________________________________ > PyOpenCL mailing list > PyOpenCL(a)tiker.net <mailto:PyOpenCL@tiker.net> > http://lists.tiker.net/listinfo/pyopencl >

5 years, 1 month

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

PyOpenCL December 2015