Everything should be made as simple as possible, but not simpler. (Albert Einstein)

Monday, May 25, 2015

some notes

Avoiding headache, centralizing all pieces of "somewhat important" notes =)


-----------------------------------------------------------------
For the famous "error: Unable to find vcvarsall.bat" when running pip.

Download compiler for Python 2.7. "Microsoft Visual C++ Compiler for Python 2.7"
http://www.microsoft.com/en-us/download/confirmation.aspx?id=44266

In Windows 7, after installing the "MS Visual C++ compiler", find the vcvarsall.bat in this dir, 
C:\Users\admin\AppData\Local\Programs\Common\Microsoft\Visual C++ for Python\9.0\
(in previous Windows version, probably it's in the C:\Program Files\Common Files)

Move subfolders inside \9.0\ to its parent, one level up. Do not move the vcvarsall.bat file.

In the Control Panel, System, add new System's environment variable:
VS90COMNTOOLS = C:\Users\admin\AppData\Local\Programs\Common\Microsoft\Visual C++ for Python\9.0

Go to C:\Python27\Lib\distutils\, open in editor msvc9compiler.py file. Search for function find_vcvarsall, edit following line,
productdir = os.path.join(toolsdir, os.pardir, os.pardir, "VC")
change to, 
productdir = os.path.join(toolsdir, os.pardir, os.pardir, "Visual C++ for Python")

Run pip. 

http://stackoverflow.com/questions/6551724/how-do-i-point-easy-install-to-vcvarsall-bat
http://stackoverflow.com/questions/3047542/building-lxml-for-python-2-7-on-windows/5122521
----------------------------------

To use OpenBLAS in Theano, (Windows OS)
Create .theanorc.txt file in user dir, i.e. in c:/User/NAME/
Add this lines,
[global]
floatX = float32
device=cpu

[blas]
ldflags = -LC:\YOUR_OPEN_BLAS_DIR\OpenBLAS-v0.2.14-Win32\bin -lopenblas

Check:
Run python C:\Python27\Lib\site-packages\theano/misc/check_blas.py
---------------------------------------

Installing nolearn/Lasagne,
Must have Theano first,
From  Git Bash, run...
Lasagne:
git clone https://github.com/Lasagne/Lasagne.git
cd Lasagne
pip install -r requirements.txt
python setup.py install
nolearn:
git clone https://github.com/dnouri/nolearn.git
cd nolearn
pip install -r requirements.txt
python setup.py develop
--------------------------------------------------

Problem with "permission denied" when installing XGBoost in Windows R Studio using RTools.
E.g.
installing to C:/Users/Think/Documents/R/win-library/3.1/xgboost/libs/x64
Warning in file.copy(files, dest, overwrite = TRUE) :
problem copying .\xgboost.dll to C:\Users\Think\Documents\R\win-library\3.1\xgboost\libs\x64\xgboost.dll: Permission denied

Solution: Simply run R Studio "as administrator" then install,
devtools::install_github('dmlc/xgboost',subdir='R-package')

--------------------------------------------------
R with OpenBLAS, Windows 7, 64 bit.

Test in R/RStudio with existing Rblas.dll,

m1 = matrix(rnorm(4000 * 4000), 4000, 4000)
m2 = matrix(rnorm(4000 * 4000), 4000, 4000)
m3 = m1 %*% m2   # use BLAS, quite long time..

Close R or RStudio.

Download precompiled libopenblas.dll, http://sourceforge.net/projects/openblas/files/v0.2.14/
(I use OpenBLAS-v0.2.14-Win64-int32.zip)
Open R's installation folder, default to C:\Program Files\R\R-3.2.3\bin\x64
Backup existing Rblas.dll by renaming it to e.g. Rblas-old.dll
Copy libopenblas.dll to the C:\Program Files\R\R-3.2.3\bin\x64 folder.
Rename it to Rblas.dll.

Test again in R/RStudio with new OpenBLAS's Rblas.dll,

m1 = matrix(rnorm(4000 * 4000), 4000, 4000)
m2 = matrix(rnorm(4000 * 4000), 4000, 4000)
m3 = m1 %*% m2   # use OpenBLAS, faster execution time..

--------------------------------------------------

Build OpenCV 3 + opencv_contrib from Source in Windows + Anaconda

SIFT and SURF was removed from OpenCV 3. For Ubuntu and OSX, there's a fix written in this article:

For Windows OS + Anaconda, here is my fix:
We need to build OpenCV 3 + contrib from source.
In this case, I want to use OpenCV in Python.

Requirements:
  • Visual Studio 12 (2013)
  • CMake


Installations:
  • Install CMake.
  • Extract opencv to a directory, e.g. in this example I use "opencv".
  • In case opencv is from source, create new directory "build" inside the opencv.
  • (If it's extracted from binary, there's already "build" dir).
  • Extract opencv_contrib.
Compilations:
  • Open CMake-gui.
  • Click on "Browse Source..." then locate the opencv's "sources" folder.
  • (It must be the "sources" folder, not its parent folder)
  • Click on "Browse Build..." then locate the opencv's "build" folder. 
  • Click "Configure" button.
  • For the first time, it will open a dialog to select compiler.
  • In this case, I use Visual Studio 12 Win64, so select the one appropriate to yours.
  • Choose "Specify native compilers".
  • Click Next.
  • Locate the compiler. Find your Visual Studio installation.
  • The compiler's default location is,
  • "Program Files/Visual Studio 12.0/VC/bin/cl.exe"
  • So, choose it for both "C" and "C++" compilers.
  • Click Finish. 
  • Wait for some minutes to finish the configuration.
  • Now, to add the contrib, find "OPENCV_EXTRA_MODULES_PATH".
  • Locate the opencv_contrib directory, and choose the "modules" dir.
Example, in my case = "D:/git/opencv_contrib/modules"

  • Find "INSTALL_C_EXAMPLES", deselect it because some people reports it to cause error.
  • Find "ENABLE_SOLUTION_FOLDERS", because it's not supported in Visual Studio.
For detail pictures, please read this doc,

  • Click "Generate" button. Results is in the opencv's "build" directory.

  • Now, open Visual Studio IDE, better to run it "as administrator".
  • In the IDE, open "OpenCV.sln" from the build dir.
  • Choose Build | Configuration Manager, choose "Release".
  • In the Solution Explorer, do right-click on the "ALL_BUILD", then "build".
  • Upon finishing, do right-click on the "INSTALL", then "build".
  • Now, OpenCV-Python should be installed.

In the Anaconda's folder, in "Lib/site-packages/" there should be "cv2.pyd" file.

To test the installation,
Open Python interpreter (or IDLE), enter "import cv2".

>>> import cv2
>>> cv2.xfeatures2d.SIFT_create()

>>> cv2.xfeatures2d.SURF_create()





Error:
In case of error happened, "ImportError: DLL load failed: The specified module could not be found.",
then try this,
(1)
Install Microsoft Visual Studio's redistributable package,
VS 2015,
VS 2013,
(2)
Try to add new environment variable "OPENCV_DIR".
-In Control Panel, open System | Advanced System Settings,
-Then, in the System Variables, add new variable OPENCV_DIR,
with value = "D:\opencv\build\install\x64\vc12".
(directory depends on your extraction location of opencv.)
-Add entry in System's PATH, with value = "D:\opencv\build\install\x64\vc12\bin".
(3)
If those steps doesn't solve the error, maybe it needs to manually install following DLLs,
API-MS-WIN-APPMODEL-RUNTIME-L1-1-0.DLL
API-MS-WIN-CORE-WINRT-ERROR-L1-1-0.DLL
API-MS-WIN-CORE-WINRT-L1-1-0.DLL
API-MS-WIN-CORE-WINRT-ROBUFFER-L1-1-0.DLL
API-MS-WIN-CORE-WINRT-STRING-L1-1-0.DLL
API-MS-WIN-SHCORE-SCALING-L1-1-1.DLL
DCOMP.DLL
IESHIMS.DLL


--------------------------------------------------
Interactive WebGL + JS cross-browser, the Mozilla Firefox browser issue.
As of Firefox v36+ to v38.0.5 (as of today 2 July 2015), there is memory leak problem with WebGL rendering.
https://bugzilla.mozilla.org/show_bug.cgi?id=1137251
http://forums.mozillazine.org/viewtopic.php?f=38&t=2939357

Temporary workaround, until there's bug fix, downgrade to v35.0.1,
http://ftp.mozilla.org/pub/mozilla.org/firefox/releases/35.0.1/

JS event handler problem with Firefox. An example:
In Chrome browser we can use this script without problem,
    document.getElementById("slider").onchange = function() {
        numTimesToSubdivide = event.srcElement.value;
    };

But it's erroneous in Firefox. "ReferenceError: event is not defined"
 To fix it for Firefox browser, modify script...
 
    document.getElementById("slider").onchange = function(event) {
        var targ;
        if (event.target) targ = event.target;
        else if (event.srcElement) targ = event.srcElement;
        if (targ.nodeType == 3) // defeat Safari bug
              targ = targ.parentNode;
        numTimesToSubdivide = targ.value;
    };
--------------------------------------------------

Problem with npm install -g bower do nothing. (http://bower.io/)
Edit the TEMP environment variable, works magically!
http://stackoverflow.com/questions/28384040/npm-install-does-nothing-how-make-it-work
https://github.com/npm/npm/issues/7318#issuecomment-95031855

--------------------------------------------------

...