{"id":635,"date":"2025-07-07T16:37:09","date_gmt":"2025-07-07T16:37:09","guid":{"rendered":"https:\/\/ni.cmu.edu\/computing\/?post_type=ht_kb&#038;p=635"},"modified":"2025-07-09T20:44:17","modified_gmt":"2025-07-09T20:44:17","slug":"triton-and-cuda-compatibility","status":"publish","type":"ht_kb","link":"https:\/\/ni.cmu.edu\/computing\/knowledge-base\/triton-and-cuda-compatibility\/","title":{"rendered":"Triton and CUDA Compatibility"},"content":{"rendered":"<h3 data-start=\"201\" data-end=\"218\"><strong data-start=\"205\" data-end=\"218\">Triton<\/strong><\/h3>\n<p data-start=\"219\" data-end=\"499\">Triton is a deep learning framework and a compiler developed by OpenAI. Its primary goal is to enable the development of high-performance machine learning models, particularly for GPU (Graphics Processing Unit) acceleration, without having to write complex CUDA code.<\/p>\n<p data-start=\"501\" data-end=\"782\">Triton allows developers to write custom kernels (small, specialized programs that run on the GPU) in a high-level, Python-like syntax, making it easier to work with GPU hardware without needing deep knowledge of the low-level CUDA programming. Some key features of Triton include:<\/p>\n<ul data-start=\"784\" data-end=\"1249\">\n<li data-start=\"784\" data-end=\"918\">\n<p data-start=\"786\" data-end=\"918\">Optimized Kernels: It can automatically optimize custom kernels to efficiently use GPU resources, offering significant speedups.<\/p>\n<\/li>\n<li data-start=\"919\" data-end=\"1073\">\n<p data-start=\"921\" data-end=\"1073\">Custom Operators: Triton enables you to define your own machine learning operators (functions or layers) that are more efficient than standard ones.<\/p>\n<\/li>\n<li data-start=\"1074\" data-end=\"1249\">\n<p data-start=\"1076\" data-end=\"1249\">Memory and Thread Management: It simplifies the complex process of memory management and thread synchronization that is typically required for efficient GPU computation.<\/p>\n<\/li>\n<\/ul>\n<p data-start=\"1251\" data-end=\"1446\">Essentially, Triton focuses on custom high-performance GPU programming for machine learning tasks, where optimizing specific operations can result in substantial performance improvements.<\/p>\n<p>Triton requires features that are found only in newer GPUs, specifically those built after the Pascal architecture (post-2016).<\/p>\n<p>Several nodes in our cluster currently have older GPUs with a Compute Capability of 6.1 or lower, which users reported is causing the compiler to fail on those systems.<\/p>\n<h6>CUDA capability of GPUs on the cluster where &#8220;Compute Capability&#8221; shows the Cuda versions it supports.<\/h6>\n<pre><strong>NODE(s)               <\/strong>| <strong>GPU Card                  <\/strong> | <strong>Version<\/strong> \r\nmind-1-1, 3, 5        | NVIDIA GeForce GTX TITAN X | 5.2\r\nmind-1-7, 9, 11, 19   | NVIDIA TITAN X (Pascal)    | 6.1\r\nmind-1-13             | NVIDIA GeForce GTX 1080 Ti | 6.1\r\nmind-0-18, 20, 22, 24 | NVIDIA GeForce RTX 2080 Ti | 7.5\r\nmind-1-24             | NVIDIA TITAN RTX           | 7.5\r\nmind-0-26             | NVIDIA GeForce RTX 3090    | 8.6\r\nmind-0-28             | NVIDIA RTX A5000           | 8.6\r\nmind-1-15             | NVIDIA L40S                | 8.9<code class=\"whitespace-pre!\"><\/code><\/pre>\n<p>You should target nodes for jobs that require Triton will run without issues on nodes equipped with GPUs that have a Compute Capability of 7.5 or higher.<\/p>\n<h6>Compatible Nodes with CUDA Compute Capability 7.5 or newer:<\/h6>\n<pre>mind-0-20 (RTX 2080 Ti, CUDA 7.5)\r\nmind-0-24 (RTX 2080 Ti, CUDA 7.5)\r\nmind-0-18 (RTX 2080 Ti, CUDA 7.5)\r\nmind-0-22 (RTX 2080 Ti, CUDA 7.5)\r\nmind-1-24 (TITAN RTX, CUDA 7.5)\r\nmind-0-26 (RTX 3090, CUDA 8.6)\r\nmind-0-28 (RTX A5000, CUDA 8.6)\r\nmind-1-15 (L40S, CUDA 8.9)<\/pre>\n<p>In your SLURM request for nodes use the nodelist option to target the nodes that support Triton:<\/p>\n<pre>--nodelist=mind-0-20,mind-0-24,mind-0-18,mind-0-22,mind-1-24,mind-0-26,mind-0-28,mind-1-15<\/pre>\n","protected":false},"excerpt":{"rendered":"<p>Triton Triton is a deep learning framework and a compiler developed by OpenAI. Its primary goal is to enable the development of high-performance machine learning models, particularly for GPU (Graphics Processing Unit) acceleration, without having to write complex CUDA code. Triton allows developers to write custom kernels (small, specialized programs&#8230;<\/p>\n","protected":false},"author":1,"comment_status":"closed","ping_status":"closed","template":"","format":"standard","meta":{"footnotes":""},"ht-kb-category":[7],"ht-kb-tag":[],"class_list":["post-635","ht_kb","type-ht_kb","status-publish","format-standard","hentry","ht_kb_category-cluster"],"jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/ni.cmu.edu\/computing\/wp-json\/wp\/v2\/ht-kb\/635","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/ni.cmu.edu\/computing\/wp-json\/wp\/v2\/ht-kb"}],"about":[{"href":"https:\/\/ni.cmu.edu\/computing\/wp-json\/wp\/v2\/types\/ht_kb"}],"author":[{"embeddable":true,"href":"https:\/\/ni.cmu.edu\/computing\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/ni.cmu.edu\/computing\/wp-json\/wp\/v2\/comments?post=635"}],"version-history":[{"count":7,"href":"https:\/\/ni.cmu.edu\/computing\/wp-json\/wp\/v2\/ht-kb\/635\/revisions"}],"predecessor-version":[{"id":642,"href":"https:\/\/ni.cmu.edu\/computing\/wp-json\/wp\/v2\/ht-kb\/635\/revisions\/642"}],"wp:attachment":[{"href":"https:\/\/ni.cmu.edu\/computing\/wp-json\/wp\/v2\/media?parent=635"}],"wp:term":[{"taxonomy":"ht_kb_category","embeddable":true,"href":"https:\/\/ni.cmu.edu\/computing\/wp-json\/wp\/v2\/ht-kb-category?post=635"},{"taxonomy":"ht_kb_tag","embeddable":true,"href":"https:\/\/ni.cmu.edu\/computing\/wp-json\/wp\/v2\/ht-kb-tag?post=635"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}