Re: [PyOpenCL] PyOpenCL on Windows
by Andreas Kloeckner
Zac Diggum <Diggum(a)gmx.de> writes:
> You made it!
> Today I saw v2015.2.4 on Christoph Gohlke's website. Great work! Thank
> you Andreas, Christoph, and Santa.
You're welcome! Thanks for providing the initial impulse to get this done.
> One last question: is it possible to build the new version for Python
> 2.7 as well? If not, I might just have found my final reason to switch
> to 3.5.
>From PyOpenCL's perspective, there shouldn't be an obstacle to building
for 2.7, but there may be something Windows-specific lurking that I'm
unaware of. At any rate--I'm enjoying using 3.5 for my day-to-day
work. It's better in a lot of little ways than Py2.
> merry Christmas / fröhliche Weihnachten...
To you too!
Andreas
3 years, 8 months
What's the point of ENABLE_GL as an option?
by Keith Brafford
Is there a good reason why ENABLE_GL is an option? If I have PyOpenCL
compiled with OpenGL support but I choose not to use it, isn't that the same
thing as having ENABLE_GL as False?
--Keith
3 years, 9 months
Build error on Win 7 x64 using Microsoft Visual C++ Compiler for Python
by Prashant Saxena
Hi,
pyopencl-master(Downloaded from github)Win7 x64Microsoft Visual C++ Compiler for PythonPython 2.7.2Nvidia Cuda 7.5
Commands:python configure.pysiteconf.py:CL_TRACE = False
CL_ENABLE_GL = True
CL_INC_DIR = ["C:\\Program Files\\NVIDIA GPU Computing Toolkit\\CUDA\\v7.5\\include"]
CL_LIB_DIR = ["C:\\Program Files\\NVIDIA GPU Computing Toolkit\\CUDA\\v7.5\\lib\\x64"]
CL_LIBNAME = ['OpenCL']
CXXFLAGS = ['-std=c++0x']
LDFLAGS = []python.exe setup.py build_ext --compiler=msvc
Errors:generating cffi module 'build\\temp.win-amd64-2.7\\Release\\pyopencl._cffi.cpp'
building 'pyopencl._cffi' extension
C:\Users\john\AppData\Local\Programs\Common\Microsoft\Visual C++ for Python\9.0\
VC\Bin\amd64\cl.exe /c /nologo /Ox /MD /W3 /GS- /DNDEBUG -DPYGPU_PACKAGE=pyopenc
l -DPYGPU_PYOPENCL=1 -DHAVE_GL=1 "-IC:\Program Files\NVIDIA GPU Computing Toolki
t\CUDA\v7.5\include" -Isrc/c_wrapper/ -IC:\Python27\include -IC:\Python27\PC /Tp
build\temp.win-amd64-2.7\Release\pyopencl._cffi.cpp /Fobuild\temp.win-amd64-2.7\
Release\build\temp.win-amd64-2.7\Release\pyopencl._cffi.obj -std=c++0x
cl : Command line warning D9002 : ignoring unknown option '-std=c++0x'
pyopencl._cffi.cpp
C:\Users\john\AppData\Local\Programs\Common\Microsoft\Visual C++ for Python\9.0\
VC\Bin\amd64\cl.exe /c /nologo /Ox /MD /W3 /GS- /DNDEBUG -DPYGPU_PACKAGE=pyopenc
l -DPYGPU_PYOPENCL=1 -DHAVE_GL=1 "-IC:\Program Files\NVIDIA GPU Computing Toolki
t\CUDA\v7.5\include" -Isrc/c_wrapper/ -IC:\Python27\include -IC:\Python27\PC /Tp
src/c_wrapper/wrap_cl.cpp /Fobuild\temp.win-amd64-2.7\Release\src/c_wrapper/wrap
_cl.obj -std=c++0x
cl : Command line warning D9002 : ignoring unknown option '-std=c++0x'
wrap_cl.cpp
d:\pyopencl-master\src\c_wrapper\wrap_cl_core.h(53) : error C2061: syntax error
: identifier 'uint32_t'
d:\pyopencl-master\src\c_wrapper\wrap_cl_core.h(59) : error C2065: 'add' : undec
lared identifier
d:\pyopencl-master\src\c_wrapper\wrap_cl_core.h(59) : error C2143: syntax error
: missing ')' before 'const'
d:\pyopencl-master\src\c_wrapper\wrap_cl_core.h(59) : error C2059: syntax error
: ')'
d:\pyopencl-master\src\c_wrapper\wrap_cl_core.h(64) : error C2061: syntax error
: identifier 'uint32_t'
d:\pyopencl-master\src\c_wrapper\wrap_cl_core.h(66) : error C2061: syntax error
: identifier 'uint32_t'
d:\pyopencl-master\src\c_wrapper\wrap_cl_core.h(70) : error C2061: syntax error
: identifier 'uint32_t'
d:\pyopencl-master\src\c_wrapper\wrap_cl_core.h(101) : error C2061: syntax error
: identifier 'uint32_t'
It would be great if we'll be able to build pyopencl using "Microsoft Visual C++ Compiler for Python".
CheersPrashant
3 years, 9 months
Import error on Ubuntu 14.04
by Prashant Saxena
Hi,pyopencl-masterUbuntu x86_64 14.04Nvidia GTX 750 TiGCC 4.8.4
siteconf.pyCL_TRACE = False
CL_ENABLE_GL = False
CL_INC_DIR = ['usr/local/cuda-7.5/include']
CL_LIB_DIR = ['usr/local/cuda-7.5/lib64']
CL_LIBNAME = ['OpenCL']
CXXFLAGS = ['-std=c++0x']
LDFLAGS = []
After building and installing I'm getting this error: import pyopencl as cl
File "/usr/local/lib/python2.7/dist-packages/pyopencl-2015.2.3-py2.7-linux-x86_64.egg/pyopencl/__init__.py", line 35, in <module>
import pyopencl.cffi_cl as _cl
File "/usr/local/lib/python2.7/dist-packages/pyopencl-2015.2.3-py2.7-linux-x86_64.egg/pyopencl/cffi_cl.py", line 37, in <module>
from .compyte.array import f_contiguous_strides, c_contiguous_strides
ImportError: No module named compyte.array
Secondly If I switch on:CL_ENABLE_GL = TrueHow and where do I inlude 'GL' library?
CheersPrashant
3 years, 9 months
Re: [PyOpenCL] PyOpenCL on Windows
by Andreas Kloeckner
Christoph Gohlke <cgohlke(a)uci.edu> writes:
> <lists(a)weasel.tiker.net> bounced. Resending to <andreask(a)illinois.edu>.
>
>
> Hi Andreas,
>
> sorry for the delay.
>
> I tried building PyOpenCL 2015.2.x on Windows and noticed several issues:
>
>
> 1. The PyPI source distribution is missing src/c_wrapper/mingw-std-threads.
>
> Using git master for now.
Fixed. [ea3d7a6]
> 2. Setup.py fails if git not found.
>
> Using this workaround:
>
> diff --git a/aksetup_helper.py b/aksetup_helper.py
> index b7378b5..29ed40a 100644
> --- a/aksetup_helper.py
> +++ b/aksetup_helper.py
> @@ -703,6 +703,7 @@ def substitute(substitutions, fname):
> def _run_git_command(cmd):
> git_error = None
> from subprocess import Popen, PIPE
> + stdout = None
> try:
> popen = Popen(["git"] + cmd, stdout=PIPE)
> stdout, stderr = popen.communicate()
> @@ -722,8 +723,11 @@ def _run_git_command(cmd):
> print(git_error)
> print("Hit Ctrl-C now if you'd like to think about the
> situation.")
> print(DASH_SEPARATOR)
> - count_down_delay(delay=5)
> - return stdout.decode("utf-8"), git_error
> + count_down_delay(delay=0)
> + if stdout:
> + return stdout.decode("utf-8"), git_error
> + else:
> + return '', None
>
>
> def check_git_submodules():
Merged, thanks. [e6ee3ca]
> 3. PyOpenCL 2015.2.x was apparently rewritten specifically for gcc
> compilers. The default platform compilers for Python on Windows, Visual
> Studio 2008, 2010, and 2015 no longer compile the code. Same for Intel
> Compilers 2013 and 2016. Too many different errors to report.
It uses a healthy dose of C++11. Clang and g++ are fine, FWIW, but I can
see why MSVC and Intel are unhappy. :/
I don't know how ready MS's (recent) Clang frontend for their compilers
is, but that may be a viable solution.
In the meantime, I've mentioned mingwpy in the README [5f08e13].
> The workaround is to use mingwpy compilers. However, they do not support
> Python 3.5, I can not debug the builds, and cffi might have issues:
>
> pyopencl._cffi.cpp:365:68: warning: unknown conversion type character
> 'z' in format
This is spurious. The code causing this looks like this:
_CFFI_UNUSED_FN
static PyObject **_cffi_unpack_args(PyObject *args_tuple, Py_ssize_t expected,
const char *fnname)
{
if (PyTuple_GET_SIZE(args_tuple) != expected) {
PyErr_Format(PyExc_TypeError,
"%.150s() takes exactly %zd arguments (%zd given)",
fnname, expected, PyTuple_GET_SIZE(args_tuple));
return NULL;
}
return &PyTuple_GET_ITEM(args_tuple, 0); /* pointer to the first item,
the others follow */
}
and %zd is allowed in PyErr_Format:
https://docs.python.org/2/c-api/string.html#c.PyString_FromFormat
> 4. strdup failed to link:
>
> wrap_cl.cpp:(.text+0x42f): undefined reference to `strdup'
>
> Using _strdup instead:
>
> diff --git a/src/c_wrapper/wrap_cl.h b/src/c_wrapper/wrap_cl.h
> index 41e57c2..576de6c 100644
> --- a/src/c_wrapper/wrap_cl.h
> +++ b/src/c_wrapper/wrap_cl.h
> @@ -31,6 +31,7 @@
> #if defined(_WIN32)
> #define NOMINMAX
> #include <windows.h>
> +#define strdup _strdup
> #else
> #include <unistd.h>
> #endif
Merged, thanks. [971ad78]
> 5. CL_KERNEL_ARG_TYPE_const' was not declared in this scope:
>
> src/c_wrapper/wrap_constants.cpp:8:24: error: 'CL_KERNEL_ARG_TYPE_const'
> was not declared in this scope
>
> Disabling CL_KERNEL_ARG_TYPE_const for now:
>
> diff --git a/src/c_wrapper/wrap_constants.cpp
> b/src/c_wrapper/wrap_constants.cpp
> index 0685b1f..661d7e8 100644
> --- a/src/c_wrapper/wrap_constants.cpp
> +++ b/src/c_wrapper/wrap_constants.cpp
> @@ -613,7 +614,7 @@ void populate_constants(void(*add)(const char*,
> const char*, int64_t value))
> // kernel_arg_type_qualifier
> #if PYOPENCL_CL_VERSION >= 0x1020
> ADD_ATTR("kernel_arg_type_qualifier", KERNEL_ARG_TYPE_, NONE);
> - ADD_ATTR("kernel_arg_type_qualifier", KERNEL_ARG_TYPE_, CONST);
> + //ADD_ATTR("kernel_arg_type_qualifier", KERNEL_ARG_TYPE_, CONST);
> ADD_ATTR("kernel_arg_type_qualifier", KERNEL_ARG_TYPE_, RESTRICT);
> ADD_ATTR("kernel_arg_type_qualifier", KERNEL_ARG_TYPE_, VOLATILE);
> #endif
It's weird that that would be missing while the others are there, but so
be it. I've made an #ifdef gate to check whether it's there. [7a9da07]
> 6. 32 bit build fails with:
>
> src/c_wrapper/error.h:255:35: error: invalid conversion from 'cl_int
> (__attribute__((__stdcall__)) *)(cl_mem) {aka int
> (__attribute__((__stdcall__)) *)(_cl_mem*)}' to 'cl_int (*)(_cl_mem*)
> {aka int (*)(_cl_mem*)}' [-fpermissive]
> call_guarded(func, #func, args)
Missing __stdcall in function pointer. Worked around. [d19d94cf]
> 7. 64 bit builds succeed and pyopencl can be imported, but running any
> test results in immediate segfaults in _cffi.pyd.
>
> I used latest CUDA and Intel OpenCL SDKs. CFFI is version 1.3.1,
> compiled with Visual Studio. I'll try to debug the crash when I find the
> time.
Hard for me to investigate without a Windows build environment. If I can help
somehow, please let me know.
I'll probably push out a release of these fixes just to prevent some
grief for folks trying to build on Windows.
> Greetings to Urbana-Champaign. I worked at Loomis Lab for ~8 years.
:) I can see the place from my office in CS right now. Hope you enjoyed
it here!
Andreas
3 years, 9 months
Re: [PyOpenCL] PyOpenCL on Windows
by Andreas Kloeckner
Ugh, so many issues. Sorry about that, and thanks for taking the time to
list them. I'll work on fixing them. CC'ing the list so they're staying
informed on what's up.
Andreas
Christoph Gohlke <cgohlke(a)uci.edu> writes:
> <lists(a)weasel.tiker.net> bounced. Resending to <andreask(a)illinois.edu>.
>
>
> Hi Andreas,
>
> sorry for the delay.
>
> I tried building PyOpenCL 2015.2.x on Windows and noticed several issues:
>
>
> 1. The PyPI source distribution is missing src/c_wrapper/mingw-std-threads.
>
> Using git master for now.
>
>
> 2. Setup.py fails if git not found.
>
> Using this workaround:
>
> diff --git a/aksetup_helper.py b/aksetup_helper.py
> index b7378b5..29ed40a 100644
> --- a/aksetup_helper.py
> +++ b/aksetup_helper.py
> @@ -703,6 +703,7 @@ def substitute(substitutions, fname):
> def _run_git_command(cmd):
> git_error = None
> from subprocess import Popen, PIPE
> + stdout = None
> try:
> popen = Popen(["git"] + cmd, stdout=PIPE)
> stdout, stderr = popen.communicate()
> @@ -722,8 +723,11 @@ def _run_git_command(cmd):
> print(git_error)
> print("Hit Ctrl-C now if you'd like to think about the
> situation.")
> print(DASH_SEPARATOR)
> - count_down_delay(delay=5)
> - return stdout.decode("utf-8"), git_error
> + count_down_delay(delay=0)
> + if stdout:
> + return stdout.decode("utf-8"), git_error
> + else:
> + return '', None
>
>
> def check_git_submodules():
>
>
> 3. PyOpenCL 2015.2.x was apparently rewritten specifically for gcc
> compilers. The default platform compilers for Python on Windows, Visual
> Studio 2008, 2010, and 2015 no longer compile the code. Same for Intel
> Compilers 2013 and 2016. Too many different errors to report.
>
> The workaround is to use mingwpy compilers. However, they do not support
> Python 3.5, I can not debug the builds, and cffi might have issues:
>
> pyopencl._cffi.cpp:365:68: warning: unknown conversion type character
> 'z' in format
>
>
> 4. strdup failed to link:
>
> wrap_cl.cpp:(.text+0x42f): undefined reference to `strdup'
>
> Using _strdup instead:
>
> diff --git a/src/c_wrapper/wrap_cl.h b/src/c_wrapper/wrap_cl.h
> index 41e57c2..576de6c 100644
> --- a/src/c_wrapper/wrap_cl.h
> +++ b/src/c_wrapper/wrap_cl.h
> @@ -31,6 +31,7 @@
> #if defined(_WIN32)
> #define NOMINMAX
> #include <windows.h>
> +#define strdup _strdup
> #else
> #include <unistd.h>
> #endif
>
>
> 5. CL_KERNEL_ARG_TYPE_const' was not declared in this scope:
>
> src/c_wrapper/wrap_constants.cpp:8:24: error: 'CL_KERNEL_ARG_TYPE_const'
> was not declared in this scope
>
> Disabling CL_KERNEL_ARG_TYPE_const for now:
>
> diff --git a/src/c_wrapper/wrap_constants.cpp
> b/src/c_wrapper/wrap_constants.cpp
> index 0685b1f..661d7e8 100644
> --- a/src/c_wrapper/wrap_constants.cpp
> +++ b/src/c_wrapper/wrap_constants.cpp
> @@ -613,7 +614,7 @@ void populate_constants(void(*add)(const char*,
> const char*, int64_t value))
> // kernel_arg_type_qualifier
> #if PYOPENCL_CL_VERSION >= 0x1020
> ADD_ATTR("kernel_arg_type_qualifier", KERNEL_ARG_TYPE_, NONE);
> - ADD_ATTR("kernel_arg_type_qualifier", KERNEL_ARG_TYPE_, CONST);
> + //ADD_ATTR("kernel_arg_type_qualifier", KERNEL_ARG_TYPE_, CONST);
> ADD_ATTR("kernel_arg_type_qualifier", KERNEL_ARG_TYPE_, RESTRICT);
> ADD_ATTR("kernel_arg_type_qualifier", KERNEL_ARG_TYPE_, VOLATILE);
> #endif
>
>
> 6. 32 bit build fails with:
>
> src/c_wrapper/error.h:255:35: error: invalid conversion from 'cl_int
> (__attribute__((__stdcall__)) *)(cl_mem) {aka int
> (__attribute__((__stdcall__)) *)(_cl_mem*)}' to 'cl_int (*)(_cl_mem*)
> {aka int (*)(_cl_mem*)}' [-fpermissive]
> call_guarded(func, #func, args)
>
>
> 7. 64 bit builds succeed and pyopencl can be imported, but running any
> test results in immediate segfaults in _cffi.pyd.
>
> I used latest CUDA and Intel OpenCL SDKs. CFFI is version 1.3.1,
> compiled with Visual Studio. I'll try to debug the crash when I find the
> time.
>
>
> Greetings to Urbana-Champaign. I worked at Loomis Lab for ~8 years.
>
> Christoph
>
>
>
> On 12/3/2015 2:53 PM, Andreas Kloeckner wrote:
>> Christoph,
>>
>> Zac Diggum (cc'd) reported that you had some trouble getting PyOpenCl to
>> compile on Windows. That makes sense since the 2015.2 series is
>> essentially a rewrite of the wrapper. I'd be interested in hearing what
>> trouble you ran into, and if I can somehow help getting it to
>> work. Could you report what issues you ran into?
>>
>> Thanks!
>> Andreas
>>
>
3 years, 9 months
PyOpenCL on Windows
by Andreas Kloeckner
Christoph,
Zac Diggum (cc'd) reported that you had some trouble getting PyOpenCl to
compile on Windows. That makes sense since the 2015.2 series is
essentially a rewrite of the wrapper. I'd be interested in hearing what
trouble you ran into, and if I can somehow help getting it to
work. Could you report what issues you ran into?
Thanks!
Andreas
3 years, 9 months
PyOpenCL 2015.2 Windows
by Zac Diggum
Hi there,
today I asked Christoph Gohlke, who runs a website offering precompiled
Python packages for Windows OS (
http://www.lfd.uci.edu/~gohlke/pythonlibs/ ), to update to PyOpenCL 2015.2.
He told me he wasn't able to compile versions newer than 2015.1 on
Windows using Visual Studio 2008, 2010 & 2015, Intel 2013 & 2016, or
mingwpy. Any ideas?
3 years, 9 months
Re: [PyOpenCL] OpenGL corruption with OpenCL and OpenGL in separate threads
by Henry Gomersall
Yes, this is plausible, but I'm a little doubtful - it seems rather too
repeatable a problem (the same lines seem to get corrupted each time,
and not in a spurious way, but in a drop-out way. That said, I'll keep
it in mind during my investigations!
I'm inclined at the moment to think it could be the beignet drivers.
I've found in the past they don't always behave quite right, and it
could easily be those.
Thanks,
Henry
On 25/11/15 18:54, CRV§ADER//KY wrote:
> I suggest you check your opencl code for buffer overflows. Not sure how
> memory segmentation works in VRAM but I would expect two threads of the
> same process to be able to freely write to each other's memory, just
> like in regular RAM.
>
> Try replacing your opencl code with something trivial (that simply keeps
> the gpu busy) and see if the problem disappears.
>
> On 25 Nov 2015 5:28 pm, "Karl Czajkowski" <karlcz(a)isi.edu
> <mailto:karlcz@isi.edu>> wrote:
>
> On Nov 25, Henry Gomersall modulated:
>
> > The OpenGL context is created independently of the OpenCL context and
> > exists in a different thread. I'm currently making no attempt to share
> > data between OpenGL and OpenCL - all the data passes via the CPU.
> >
>
> How is this sharing via CPU done? Are you copying data into opengl
> vertex buffers or textures via opengl API calls? Are these calls done
> in the opengl thread at the right point in the rendering loop?
>
> At the very least, you need to do something to synchronize your reads
> of results from the opencl thread and your writes to the opengl
> thread. I would expect you'd copy opencl results into Python during
> the opencl thread (between kernel runs) and stick them in a
> synchronized Python structure to then pull them out and send to opengl
> from the opengl thread.
>
>
> > I'm finding that I'm getting substantial artifacts in the OpenGL
> > rendering. Specifically, I get a heavy flickering of the rendering
> - it
> > looks like e.g. certain line segments are not being drawn (it's a
> simple
> > line drawing program) for some frame.
> >
>
> Is the renderer using line vertices copied over from opencl results?
> Do the missing lines make sense as rendering a partially copied array?
>
> Also, are you using double-buffering in opengl so that only completely
> rendered frames will be swapped to the display? Otherwise, you could
> also see parts of the scene drawn or missing depending on when they
> happen relative to the vertical blanking period and buffer clearing.
>
>
> > Curiously, it doesn't break _every_ run. About 1 run in 10 works
> > perfectly, with no visual problems at all. I wonder if there is some
> > race condition causing a (setup?) problem.
> >
>
> This could be dumb luck that the workers get phase-locked and
> synchronized by accident.
>
>
> Karl
>
>
> _______________________________________________
> PyOpenCL mailing list
> PyOpenCL(a)tiker.net <mailto:PyOpenCL@tiker.net>
> http://lists.tiker.net/listinfo/pyopencl
>
3 years, 9 months
