{"id":103,"date":"2020-01-22T19:39:04","date_gmt":"2020-01-22T19:39:04","guid":{"rendered":"https:\/\/ni.cmu.edu\/computing\/?post_type=ht_kb&#038;p=103"},"modified":"2020-02-18T19:28:38","modified_gmt":"2020-02-18T19:28:38","slug":"matlab","status":"publish","type":"ht_kb","link":"https:\/\/ni.cmu.edu\/computing\/knowledge-base\/matlab\/","title":{"rendered":"Matlab"},"content":{"rendered":"<p><a href=\"http:\/\/www.mathworks.co.uk\" target=\"_blank\" rel=\"noopener noreferrer\">Matlab\u00a0<\/a> is a numerical computing and programming environment with a broad range of functionality (matrix manipulation, numerical linear algebra, general-purpose graphics, etc.). Additionally, special application areas are served by a large number of optional toolboxes.<\/p>\n<h4>Running Matlab on the headnode<\/h4>\n<p>Users need to avoid using the head node for processing using Matlab or other software. However, Matlab Distributed Computing toolbox\/server is installed on the cluster and this requires running Matlab on the head node due to how it integrates with SLURM.<\/p>\n<h4 id=\"submissionScripts\">How do I run Matlab graphically on the cluster?<\/h4>\n<p>X Window System (X11) server will be needed on your local computer.\u00a0 How you get this will depend on your local computer&#8217;s operating system and your preferences.\u00a0 This is covered in more detail in the article <a href=\"\/knowledge-base\/logging-onto-the-cluster\/\">Logging onto the Cluster.<\/a><\/p>\n<p>Users who want to use Matlab graphically and display it on their local computers will need to schedule an interactive session in SLURM using a similar command to this. \u00a0 In is important to include &#8211;x11 and adjust the request to the necessary resources (queue, # cpus, memory and time ) you will need for your interactive session.<\/p>\n<pre>srun --x11 -p cpu --cpus-per-task=1 --mem=10GB --time=4:00:00 --pty $SHELL<\/pre>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-large wp-image-313\" src=\"https:\/\/ni.cmu.edu\/computing\/wp-content\/uploads\/2020\/02\/mind-matlab-gui-897x1024.jpg\" alt=\"\" width=\"643\" height=\"734\" srcset=\"https:\/\/ni.cmu.edu\/computing\/wp-content\/uploads\/2020\/02\/mind-matlab-gui-897x1024.jpg 897w, https:\/\/ni.cmu.edu\/computing\/wp-content\/uploads\/2020\/02\/mind-matlab-gui-263x300.jpg 263w, https:\/\/ni.cmu.edu\/computing\/wp-content\/uploads\/2020\/02\/mind-matlab-gui-768x877.jpg 768w, https:\/\/ni.cmu.edu\/computing\/wp-content\/uploads\/2020\/02\/mind-matlab-gui-44x50.jpg 44w, https:\/\/ni.cmu.edu\/computing\/wp-content\/uploads\/2020\/02\/mind-matlab-gui-53x60.jpg 53w, https:\/\/ni.cmu.edu\/computing\/wp-content\/uploads\/2020\/02\/mind-matlab-gui-88x100.jpg 88w, https:\/\/ni.cmu.edu\/computing\/wp-content\/uploads\/2020\/02\/mind-matlab-gui.jpg 1160w\" sizes=\"auto, (max-width: 643px) 100vw, 643px\" \/><\/p>\n<p>The following is the command set and output from the above screen capture:<\/p>\n<pre>dpane@my-vm:~$ ssh -Y dpane@mind.cs.cmu.edu\r\ndpane@mind.cs.cmu.edu's password: \r\nLast login: Thu Feb 13 07:38:02 2020 from claymore.cnbc.cmu.edu\r\n.\r\n.\r\n[dpane@mind ~]$ \r\n[dpane@mind ~]$ srun --x11 -p cpu --cpus-per-task=1 --mem=10GB --time=4:00:00 --pty $SHELL\r\n[dpane@mind-1-21-1 ~]$ module load matlab-9.5 \r\n[dpane@mind-1-21-1 ~]$ matlab\r\nMATLAB is selecting SOFTWARE OPENGL rendering.\r\nGtk-Message: 13:05:49.875: Failed to load module \"canberra-gtk-module\"\r\nexit\r\n[dpane@mind-1-21-1 ~]$ exit\r\nexit\r\nsrun: error: mind-1-21-1: task 0: Exited with exit code 1\r\n[dpane@mind ~]$ \r\n<\/pre>\n<p>If you receive &#8220;<strong>srun: error: No DISPLAY variable set, cannot setup x11 forwarding.<\/strong>&#8221; error, try logging onto the cluster using the X11 option<\/p>\n<p>[dpane@mind ~]$ srun &#8211;x11 -p cpu &#8211;cpus-per-task=1 &#8211;mem=10GB &#8211;time=4:00:00 &#8211;pty $SHELL<br \/>\nsrun: error: No DISPLAY variable set, cannot setup x11 forwarding.<\/p>\n<p>You should try using on of the following options when you remote login program using ssh.<\/p>\n<p><strong>-Y<\/strong> ==&gt;\u00a0 Enables trusted X11 forwarding. Trusted X11 forwardings are not subjected to the X11 SECURITY extension controls.<\/p>\n<h4 id=\"submissionScripts\">Non-interactive Matlab Sessions<\/h4>\n<p>Using Matlab in the interactive mode, as we explain above, can be used to take advantage of the integrated environment. However, it can also be run in a non-interactive way that is suitable for &#8220;batch processing&#8221; and will allow users to takefull advantage of cluster resources. In most cases, the preferred mode of operation for Matlab on our cluster is non-interactive.<\/p>\n<p>One way to run Matlab non-interactively is through<\/p>\n<ul>\n<li>re-directing the standard input and output when invoking Matlab and<\/li>\n<li>invoking Matlab from a submission script, submitted to the queue via the PBS scheduler.<\/li>\n<\/ul>\n<p>Input and output re-direction is an one way of running Matlab non-interactivelly. It is achieved using the Linux operators <strong>&lt;<\/strong> and <strong>&gt;<\/strong> <strong>,<\/strong> with Matlab taking a code file as an input and writing the output to a file, <em>e.g.<\/em> <strong>matlab &lt; myScript.m &gt; myOutput.txt<\/strong> <strong>. <\/strong>The main function\/program (<em>e.g.<\/em> <strong>myScript.m<\/strong>) should have the <strong>exit <\/strong>command at the end in order to force Matlab to quit after finishing the execution of the code.<\/p>\n<p>Here is an example demonstrating a non-interactive Matlab program. A program file named <code>mystats.m<\/code> that contains a main function, <code>mystats<\/code>, and two local functions, <code>mymean<\/code> and <code>mymedian<\/code>.<\/p>\n<pre>function [avg, med] = mystats(x)\r\nn = length(x);\r\navg = mymean(x,n);\r\nmed = mymedian(x,n);\r\nend\r\n\r\nfunction a = mymean(v,n)\r\n% MYMEAN Example of a local function.\r\n\r\na = sum(v)\/n;\r\nend\r\n\r\nfunction m = mymedian(v,n)\r\n% MYMEDIAN Another example of a local function.\r\n\r\nw = sort(v);\r\nif rem(n,2) == 1\r\n    m = w((n + 1)\/2);\r\nelse\r\n    m = (w(n\/2) + w(n\/2 + 1))\/2;\r\nend\r\nend<\/pre>\n<p>The job is sent to the queue and executed on a backend node using the following PBS file provided and the command <strong>qsub run.sh<\/strong>. The script <strong>run.sh<\/strong> contains the following line to run the Matlab script:<\/p>\n<pre>matlab -nodisplay -nosplash &lt; mystats.m &gt; run.log\r\n<\/pre>\n<p>The flag <strong>nodisplay<\/strong> instructs Matlab to run without the GUI, while <strong>nosplash<\/strong> prevents the display of the Matlab logo. The <strong>&lt;<\/strong> redirection operator ensures that Matlab runs the script <strong>main.m<\/strong>, while the <strong>&gt;<\/strong> operator re-directs the standard output (normally to the terminal) to <strong>run.log<\/strong> file.<\/p>\n<p>The example is run in batch mode with the command <strong>sbatch run.sh<\/strong>, using the following PBS file:<\/p>\n<pre>#!\/bin\/bash -l\r\n# Job name\r\n#SBATCH --job-name=batch_matlab_example    \r\n\r\n# Run on a single CPU\r\n#SBATCH --ntasks=1    \r\n# Submit job to cpu queue                \r\n#SBATCH -p cpu\r\n\r\n# Job memory request\r\n#SBATCH --mem=10gb\r\n# Time limit days-hrs:min:sec\r\n#SBATCH --time 00-00:05:00\r\n\r\n# Standard output and error log\r\n#SBATCH --output=\/user_data\/dpane\/exampleOut.out\r\n\r\nhostname\r\necho \"job starting\"\r\nmodule load matlab-8.6\r\ncd \/old_home\/dpane\/from_Brian\/klab_Suite2P_SLURM_201906\r\necho \"RUNNING MATLAB\"\r\nmatlab -nodisplay -nosplash &lt; main.m &gt; run.log\r\nmodule unload matlab-8.6\r\necho \"job finished\"\r\n<\/pre>\n<p>Notice how MATLAB is instructed to not load the interactive window.<\/p>\n<p><strong>Note<\/strong>: do not turn java off when lauching MATLAB (<em>i.e.<\/em> do not invoke <strong>matlab -nojvm<\/strong>); <strong>matlabpool<\/strong> uses the Java Virtual Machine.<\/p>\n<p>After the job finishes, the CPU times spent executed the loops in <strong>main.m<\/strong> can be found in <strong>timings.dat<\/strong>, showing a clear speed-up of the execution in parallel.<\/p>\n<h5>Running Matlab on parallel hardware<\/h5>\n<p>Matlab can also be run where you can take advantage of parallel hardware.\u00a0 You should refer to the Matlab <a href=\"http:\/\/Getting Started with Parallel Computing Toolbox\">documentation<\/a> for details on learning how to take advantage of these features.<\/p>\n<h4 id=\"exploitingTrivialParallelism\">Exploiting trivial parallelism<\/h4>\n<p>An easy way to exploit multi-core systems is to split the workflow into parts that can be processed completely independently. The typical example in this category is a parameter sweep, where the same Matlab script is run a large number of times using different inputs; these runs are independent from each other and can be carried out concurrently. Thus, the entire workflow can be scheduled in jobs that group 8 independent runs to match the 8 cores available per compute node. This strategy is best coupled with the use of the Matlab <strong>mcc<\/strong> compiler in order to avoid an excessive use of licenses.<\/p>\n<h4 id=\"multithreadedMexProgramming\">Multi-threaded MEX programming<\/h4>\n<p>Yet another way to exploit multi-core systems is via multi-threaded Mex programming. Mex (<strong>M<\/strong>atlab <strong>EX<\/strong>ecutable) files are dynamically linked subroutines compiled from C, C++ or Fortran source code that can be run from within Matlab in the same way as M-files or built-in functions. These guidelines assume knowledge of serial Mex programming and provide an example of how to augment serial execution with multi-threading through <a href=\"https:\/\/en.wikipedia.org\/wiki\/OpenMP\"><span class=\"twikiNewLink\">OpenMP<\/span><\/a>. Coupled with <span class=\"twikiNewLink\">OpenMP<\/span> multi-threading, Mex files become a powerful method to accelerate key parts of a Matlab program.<\/p>\n<p>The main reason to write Mex files in C or Fortran (thus abandoning the high-level abstracted Matlab programming) is to gain speed of execution in computationally intensive operations that otherwise become a bottleneck in an application. Typically, this is done to replace a function that is identified through profiling as being slow and\/or called a large number of times. Nevertheless, this programming effort is rewarded to various degrees, with the greatest relative benefits normally met when a Mex replaces a Matlab script (M-file). At the other extreme, Matlab operations that rely on performance libraries like FFTW (<em>e.g.<\/em> <strong>fftn<\/strong>) or BLAS\/LAPACK (<em>e.g.<\/em> solution of a dense linear systems, <strong>A\\b<\/strong>), which are highly optimised have nothing or very little to benefit from Mex programming. The best source for learning Mex programming is the <a href=\"https:\/\/www.mathworks.com\/matlabcentral\/answers\/345725-mex-programming-for-a-beginner?s_tid=srchtitle\">Mathworks webpages<\/a>.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Matlab\u00a0 is a numerical computing and programming environment with a broad range of functionality (matrix manipulation, numerical linear algebra, general-purpose graphics, etc.). Additionally, special application areas are served by a large number of optional toolboxes. Running Matlab on the headnode Users need to avoid using the head node for processing&#8230;<\/p>\n","protected":false},"author":1,"comment_status":"closed","ping_status":"closed","template":"","format":"standard","meta":{"footnotes":""},"ht-kb-category":[11],"ht-kb-tag":[],"class_list":["post-103","ht_kb","type-ht_kb","status-publish","format-standard","hentry","ht_kb_category-software"],"jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/ni.cmu.edu\/computing\/wp-json\/wp\/v2\/ht-kb\/103","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/ni.cmu.edu\/computing\/wp-json\/wp\/v2\/ht-kb"}],"about":[{"href":"https:\/\/ni.cmu.edu\/computing\/wp-json\/wp\/v2\/types\/ht_kb"}],"author":[{"embeddable":true,"href":"https:\/\/ni.cmu.edu\/computing\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/ni.cmu.edu\/computing\/wp-json\/wp\/v2\/comments?post=103"}],"version-history":[{"count":9,"href":"https:\/\/ni.cmu.edu\/computing\/wp-json\/wp\/v2\/ht-kb\/103\/revisions"}],"predecessor-version":[{"id":315,"href":"https:\/\/ni.cmu.edu\/computing\/wp-json\/wp\/v2\/ht-kb\/103\/revisions\/315"}],"wp:attachment":[{"href":"https:\/\/ni.cmu.edu\/computing\/wp-json\/wp\/v2\/media?parent=103"}],"wp:term":[{"taxonomy":"ht_kb_category","embeddable":true,"href":"https:\/\/ni.cmu.edu\/computing\/wp-json\/wp\/v2\/ht-kb-category?post=103"},{"taxonomy":"ht_kb_tag","embeddable":true,"href":"https:\/\/ni.cmu.edu\/computing\/wp-json\/wp\/v2\/ht-kb-tag?post=103"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}