C++ AMP : Language and Programming Model Version 1.0, August 2012 © 2012 Microsoft Corporation. All rights reserved. This specification reflects input from NVIDIA Corporation (Nvidia) and Advanced Micro Devices, Inc. (AMD). Copyright License. Microsoft grants you a license under its copyrights in the specification to (a) make copies of the specification to develop your implementation of the specification, and (b) distribute portions of the specification in your implementation or your documentation of your implementation. Patent Notice. Microsoft provides you certain patent rights for implementations of this specification under the terms of Microsoft’s Community Promise, available at http://www.microsoft.com/openspecifications/en/us/programs/community- promise/default.aspx. THIS SPECIFICATION IS PROVIDED "AS IS." MICROSOFT MAY CHANGE THIS SPECIFICATION OR ITS OWN IMPLEMENTATIONS AT ANY TIME AND WITHOUT NOTICE. MICROSOFT MAKES NO REPRESENTATIONS OR WARRANTIES, EXPRESS, IMPLIED, OR STATUTORY, (1) AS TO THE INFORMATION IN THIS SPECIFICATION, INCLUDING ANY WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, NON-INFRINGEMENT, OR TITLE; OR (2) THAT THE IMPLEMENTATION OF SUCH CONTENTS WILL NOT INFRINGE ANY THIRD PARTY PATENTS OR OTHER RIGHTS. C++ AMP : Language and Programming Model : Version 0. 2012 ABSTRACT C++ AMP (Accelerated Massive Parallelism) is a native programming model that contains elements that span the C++ programming language and its runtime library. It provides an easy way to write programs that compile and execute on data- parallel hardware, such as graphics cards (GPUs). The syntactic changes introduced by C++ AMP are minimal, but additional restrictions are enforced to reflect the limitations of data parallel hardware. Data parallel algorithms are supported by the introduction of multi-dimensional array types, array operations on those types, indexing, asynchronous memory transfer, shared memory, synchronization and tiling/partitioning techniques. 1 Overview .................................................................................................................................................................. 1 1.1 Conformance ............................................................................................................................................................ 1 1.2 Definitions ................................................................................................................................................................. 2 1.3 Error Model ............................................................................................................................................................... 4 1.4 Programming Model ................................................................................................................................................. 5 2 C++ Language Extensions for Accelerated Computing ............................................................................................... 6 2.1 Syntax ........................................................................................................................................................................ 6 2.1.1 Function Declarator Syntax .................................................................................................................................... 7 2.1.2 Lambda Expression Syntax ..................................................................................................................................... 7 2.1.3 Type Specifiers ....................................................................................................................................................... 7 2.2 Meaning of Restriction Specifiers ............................................................................................................................. 8 2.2.1 Function Definitions ............................................................................................................................................... 8 2.2.2 Constructors and Destructors ................................................................................................................................ 8 2.2.3 Lambda Expressions ............................................................................................................................................... 9 2.3 Expressions Involving Restricted Functions ............................................................................................................ 10 2.3.1 Function pointer conversions .............................................................................................................................. 10 2.3.2 Function Overloading ........................................................................................................................................... 10 2.3.2.1 Overload Resolution .................................................................................................................................... 11 2.3.2.2 Name Hiding .............................................................................................................................................. 12 2.3.3 Casting.................................................................................................................................................................. 12 2.4 amp Restriction Modifier ........................................................................................................................................ 13 2.4.1 Restrictions on Types ........................................................................................................................................... 13 2.4.1.1 Type Qualifiers ............................................................................................................................................ 13 2.4.1.2 Fundamental Types ..................................................................................................................................... 13 2.4.1.2.1 Floating Point Types ................................................................................................................................ 13 2.4.1.3 Compound Types ........................................................................................................................................ 14 2.4.2 Restrictions on Function Declarators ................................................................................................................... 14 2.4.3 Restrictions on Function Scopes .......................................................................................................................... 14 2.4.3.1 Literals ......................................................................................................................................................... 14 2.4.3.2 Primary Expressions (C++11 5.1) ................................................................................................................. 14 2.4.3.3 Lambda Expressions .................................................................................................................................... 15 2.4.3.4 Function Calls (C++11 5.2.2) ........................................................................................................................ 15 2.4.3.5 Local Declarations ....................................................................................................................................... 15 2.4.3.5.1 tile_static Variables ................................................................................................................................. 15 2.4.3.6 Type-Casting Restrictions ............................................................................................................................ 15 2.4.3.7 Miscellaneous Restrictions .......................................................................................................................... 16 3 Device Modeling ..................................................................................................................................................... 16 3.1 The concept of a compute accelerator ................................................................................................................... 16 3.2 accelerator .............................................................................................................................................................. 16 3.2.1 Default Accelerator .............................................................................................................................................. 16 3.2.2 Synopsis ............................................................................................................................................................... 17 3.2.3 Static Members .................................................................................................................................................... 18 3.2.4 Constructors ......................................................................................................................................................... 18 3.2.5 Members .............................................................................................................................................................. 19 3.2.6 Properties ............................................................................................................................................................. 20 3.3 accelerator_view..................................................................................................................................................... 21 3.3.1 Synopsis ............................................................................................................................................................... 21 3.3.2 Queuing Mode ..................................................................................................................................................... 22 3.3.3 Constructors ......................................................................................................................................................... 22 3.3.4 Members .............................................................................................................................................................. 23 3.4 Device enumeration and selection API ................................................................................................................... 24 3.4.1 Synopsis ............................................................................................................................................................... 24 4 Basic Data Elements ................................................................................................................................................ 25 4.1 index<N> ................................................................................................................................................................. 25 4.1.1 Synopsis ............................................................................................................................................................... 25 4.1.2 Constructors ......................................................................................................................................................... 27 4.1.3 Members .............................................................................................................................................................. 27 4.1.4 Operators ............................................................................................................................................................. 28 4.2 extent<N> ............................................................................................................................................................... 29 4.2.1 Synopsis ............................................................................................................................................................... 29 4.2.2 Constructors ......................................................................................................................................................... 31 4.2.3 Members .............................................................................................................................................................. 31 4.2.4 Operators ............................................................................................................................................................. 32 4.3 tiled_extent<D0,D1,D2> ......................................................................................................................................... 34 4.3.1 Synopsis ............................................................................................................................................................... 34 4.3.2 Constructors ......................................................................................................................................................... 36 4.3.3 Members .............................................................................................................................................................. 36 4.3.4 Operators ............................................................................................................................................................. 36 4.4 tiled_index<D0,D1,D2> ........................................................................................................................................... 37 4.4.1 Synopsis ............................................................................................................................................................... 38 4.4.2 Constructors ......................................................................................................................................................... 40 4.4.3 Members .............................................................................................................................................................. 40 4.5 tile_barrier .............................................................................................................................................................. 41 4.5.1 Synopsis ............................................................................................................................................................... 41 4.5.2 Constructors ......................................................................................................................................................... 41 4.5.3 Members .............................................................................................................................................................. 41 4.5.4 Other Memory Fences ......................................................................................................................................... 42 4.6 completion_future .................................................................................................................................................. 42 4.6.1 Synopsis ............................................................................................................................................................... 43 4.6.2 Constructors ......................................................................................................................................................... 43 4.6.3 Members .............................................................................................................................................................. 44 5 Data Containers ...................................................................................................................................................... 45 5.1 array<T,N> .............................................................................................................................................................. 45 5.1.1 Synopsis ............................................................................................................................................................... 45 5.1.2 Constructors ......................................................................................................................................................... 52 5.1.2.1 Staging Array Constructors.......................................................................................................................... 55 5.1.3 Members .............................................................................................................................................................. 57 5.1.4 Indexing ................................................................................................................................................................ 58 5.1.5 View Operations .................................................................................................................................................. 59 5.2 array_view<T,N> ..................................................................................................................................................... 60 5.2.1 Synopsis ............................................................................................................................................................... 61 5.2.1.1 array_view<T,N> ......................................................................................................................................... 62 5.2.1.2 array_view<const T,N> ................................................................................................................................ 65 5.2.2 Constructors ......................................................................................................................................................... 68 5.2.3 Members .............................................................................................................................................................. 69 5.2.4 Indexing ................................................................................................................................................................ 70 5.2.5 View Operations .................................................................................................................................................. 71 5.3 Copying Data ........................................................................................................................................................... 73 5.3.1 Synopsis ............................................................................................................................................................... 73 5.3.2 Copying between array and array_view .............................................................................................................. 74 5.3.3 Copying from standard containers to arrays or array_views ............................................................................... 76 5.3.4 Copying from arrays or array_views to standard containers ............................................................................... 77 6 Atomic Operations .................................................................................................................................................. 77 6.1 Synposis .................................................................................................................................................................. 77 6.2 Atomically Exchanging Values ................................................................................................................................. 78 6.3 Atomically Applying an Integer Numerical Operation ............................................................................................ 79 7 Launching Computations: parallel_for_each .......................................................................................................... 80 7.1 Capturing Data in the Kernel Function Object ........................................................................................................ 83 7.2 Exception Behaviour ............................................................................................................................................... 83 8 Correctly Synchronized C++ AMP Programs ............................................................................................................ 83 8.1 Concurrency of sibling threads launched by a parallel_for_each call..................................................................... 83 8.1.1 Correct usage of tile barriers ............................................................................................................................... 84 8.1.2 Establishing order between operations of concurrent parallel_for_each threads ................................. 85 8.1.2.1 Barrier-incorrect programs ......................................................................................................................... 86 8.1.2.2 Compatible memory operations ................................................................................................................. 86 8.1.2.3 Concurrent memory operations.................................................................................................................. 87 8.1.2.4 Racy programs ............................................................................................................................................. 88 8.1.2.5 Race-free programs ..................................................................................................................................... 88 8.2 Cumulative effects of a parallel_for_each call ........................................................................................... 88 8.3 Effects of copy and copy_async operations ............................................................................................................ 90 8.4 Effects of array_view::synchronize, synchronize_async and refresh functions ...................................................... 91 9 Math Functions ....................................................................................................................................................... 92 9.1 fast_math ................................................................................................................................................................ 92 9.2 precise_math .......................................................................................................................................................... 94 9.3 Miscellaneous Math Functions (Optional) ............................................................................................................ 101 10 Graphics (Optional) ............................................................................................................................................... 103 10.1 texture<T,N> ......................................................................................................................................................... 104 10.1.1 Synopsis ......................................................................................................................................................... 104 10.1.2 Introduced typedefs ...................................................................................................................................... 106 10.1.3 Constructing an uninitialized texture ............................................................................................................ 106 10.1.4 Constructing a texture from a host side iterator ........................................................................................... 107 10.1.5 Constructing a texture from a host-side data source .................................................................................... 108 10.1.6 Constructing a texture by cloning another .................................................................................................... 109 10.1.7 Assignment operator ..................................................................................................................................... 110 10.1.8 Copying textures ............................................................................................................................................ 110 10.1.9 Moving textures............................................................................................................................................. 110 10.1.10 Querying texture’s physical characteristics ................................................................................................... 110 10.1.11 Querying texture’s logical dimensions .......................................................................................................... 111 10.1.12 Querying the accelerator_view where the texture resides ........................................................................... 111 10.1.13 Reading and writing textures ........................................................................................................................ 111 10.1.14 Global texture copy functions ....................................................................................................................... 112 10.1.14.1 Global async texture copy functions ..................................................................................................... 112 10.1.15 Direct3d Interop Functions ............................................................................................................................ 112 10.2 writeonly_texture_view<T,N> .............................................................................................................................. 113 10.2.1 Synopsis ......................................................................................................................................................... 113 10.2.2 Introduced typedefs ...................................................................................................................................... 113 10.2.3 Construct a writeonly view over a texture .................................................................................................... 114 10.2.4 Copy constructors and assignment operators ............................................................................................... 114 10.2.5 Destructor ...................................................................................................................................................... 114 10.2.6 Querying underlying texture’s physical characteristics ................................................................................. 114 10.2.7 Querying the underlying texture’s accelerator_view .................................................................................... 114 10.2.7.1 Querying underlying texture’s logical dimensions (through a view) ........................................................ 115 10.2.7.2 Writing a write-only texture view ............................................................................................................. 115 10.2.8 Global writeonly_texture_view copy functions............................................................................................. 115 10.2.8.1 Global async writeonly_texture_view copy functions .............................................................................. 115 10.2.9 Direct3d Interop Functions ............................................................................................................................ 115 10.3 norm and unorm ................................................................................................................................................... 116 10.3.1 Synopsis ......................................................................................................................................................... 116 10.3.2 Constructors and Assignment........................................................................................................................ 117 10.3.3 Operators ....................................................................................................................................................... 118 10.4 Short Vector Types ................................................................................................................................................ 118 10.4.1 Synopsis ......................................................................................................................................................... 118 10.4.2 Constructors .................................................................................................................................................. 120 10.4.2.1 Constructors from components ................................................................................................................ 120 10.4.2.2 Explicit conversion constructors ............................................................................................................... 120 10.4.3 Component Access (Swizzling) ...................................................................................................................... 121 10.4.3.1 Single-component access .......................................................................................................................... 121 10.4.3.2 Two-component access ............................................................................................................................. 121 10.4.3.3 Three-component access .......................................................................................................................... 122 10.4.3.4 Four-component access ............................................................................................................................ 122 10.5 Template Versions of Short Vector Types ............................................................................................................. 123 10.5.1 Synopsis ......................................................................................................................................................... 123 10.5.2 short_vector<T,N> type equivalences ........................................................................................................... 125 10.6 Template class short_vector_traits ...................................................................................................................... 126 10.6.1 Synopsis ......................................................................................................................................................... 126 10.6.2 Typedefs ........................................................................................................................................................ 129 10.6.3 Members ....................................................................................................................................................... 130 11 D3D interoperability (Optional) ............................................................................................................................ 131 12 Error Handling ....................................................................................................................................................... 133 12.1 static_assert .......................................................................................................................................................... 133 12.2 Runtime errors ...................................................................................................................................................... 133 12.2.1 runtime_exception ........................................................................................................................................ 134 12.2.1.1 Specific Runtime Exceptions ..................................................................................................................... 134 12.2.2 out_of_memory ............................................................................................................................................. 134 12.2.3 invalid_compute_domain .............................................................................................................................. 135 12.2.4 unsupported_feature .................................................................................................................................... 135 12.2.5 accelerator_view_removed ........................................................................................................................... 136 12.3 Error handling in device code (amp-restricted functions) (Optional) ................................................................... 136 13 Appendix: C++ AMP Future Directions (Informative)............................................................................................. 138 13.1 Versioning Restrictions ......................................................................................................................................... 138 13.1.1 auto restriction .............................................................................................................................................. 138 13.1.2 Automatic restriction deduction ................................................................................................................... 139 13.1.3 amp Version................................................................................................................................................... 139 13.2 Projected Evolution of amp-Restricted Code........................................................................................................ 139 Page 1 1 Overview 1 2 3 C++ AMP is a compiler and programming model extension to C++ that enables the acceleration of C++ code on data-parallel 4 hardware. 5 6 One example of data-parallel hardware today is the discrete graphics card (GPU), which is becoming increasingly relevant for 7 general purpose parallel computations, in addition to its main function as a graphics accelerator. While GPUs may be tightly 8 integrated with the CPU and can share memory space, C++ AMP programmers must remain aware that the GPU can also be 9 physically separate from the CPU, having discrete memory address space, and incurring high cost for transferring data 10 between CPU and GPU memory. The programmer must carefully balance the cost of this potential data transfer overhead 11 against the computational acceleration achievable by parallel execution on the device. The programmer must also follow 12 some basic conventions to avoid unnecessary copies on systems that have separate memory (see Error! Reference source 13 not found. Error! Reference source not found. and the discard_data() method in Error! Reference source not found.). 14 15 Another example of data-parallel hardware is the SIMD vector instruction set, and associated registers, found in all modern 16 processors. 17 18 For the remainder of this specification, we shall refer to the data-parallel hardware as the accelerator. In the few places 19 where the distinction matters, we shall refer to a GPU or a VectorCPU. 20 21 The C++ AMP programming model gives the developer explicit control over all of the above aspects of interaction with the 22 accelerator. The developer may explicitly manage all communication between the CPU and the accelerator, and this 23 communication can be either synchronous or asynchronous. The data parallel computations performed on the accelerator 24 are expressed using high-level abstractions, such as multi-dimensional arrays, high level array manipulation functions, and 25 multi-dimensional indexing operations, all based on a large subset of the C++ programming language. 26 27 The programming model contains multiple layers, allowing developers to trade off ease-of-use with maximum performance. 28 29 C++ AMP is composed of three broad categories of functionality: 30 31 1. C++ language and compiler 32 a. Kernel functions are compiled into code that is specific to the accelerator. 33 2. Runtime 34 a. The runtime contains a C++ AMP abstraction of lower-level accelerator APIs, as well as support for 35 multiple host threads and processors, and multiple accelerators. 36 b. Asychronous execution is supported through an eventing model. 37 3. Programming model 38 a. A set of classes describing the shape and extent of data. 39 b. A set of classes that contain or refer to data used in computations 40 c. A set of functions for copying data to and from accelerators 41 d. A math library 42 e. An atomic library 43 f. A set of miscellaneous intrinsic functions 44 1.1 Conformance 45 46 All text in this specification falls into one of the following categories: 47  Informative: shown in this style. 48 Informative text is non-normative; for background information only; not required to be implemented in order to 49 conform to this specification. C++ AMP : Language and Programming Model : Version 0.9 : January 2012 Page 2 50  Microsoft-specific: shown in this style. 51 Microsoft-specific text is non-normative; for background information only; not required to be implemented in order 52 to conform to this specification; explains features that are specific to the Microsoft implementation of the C++ AMP 53 programming model. However, implementers are free to implement these feature, or any subset thereof. 54  Normative: all text, unless otherwise marked (see previous categories) is normative. Normative text falls into the 55 following two sub-categories: 56 o Optional: each section of the specification that falls into this sub-category includes the suffix “(Optional)” 57 in its title. A conforming implementation of C++ AMP may choose to support such features, or not. 58 (Microsoft-specific portions of the text are also Optional.) 59 o Required: unless otherwise stated, all Normative text falls into the sub-category of Required. A conforming 60 implementation of C++ AMP must support all Required features. 61 Conforming implementations shall provide all normative features and any number of optional features. Implementations may 62 provide additional features so long as these features are exposed in namespaces other than those listed in this specification. 63 Implementation may provide additional language support for amp-restricted functions (section 2.1) by following the rules set 64 forth in section 13. 65 66 The programming model utilizes Microsoft’s Visual C++ syntax for properties. Any such property shall be considered optional. 67 An implementation is free to use equivalent mechanisms for introducing such properties as long as they provide the same 68 functionality of indirection to a member function as Microsoft’s Visual C++ properties do. 69 1.2 Definitions 70 71 This section introduces terms used within the body of this specification. 72 73  Accelerator 74 A hardware device or capability that enables accelerated computation on data-parallel workloads. Examples 75 include: 76 o Graphics Processing Unit, or GPU, other coprocessor, accessible through the PCIe bus. 77 o Graphics Processing Unit, or GPU, or other coprocessor that is integrated with a CPU on the same die. 78 o SIMD units of the host node exposed through software emulation of a hardware accelerator. 79 80  Array 81 A dense N-dimensional data container. 82 83  Array View 84 A view into a contiguous piece of memory that adds array-like dimensionality. 85 86  Compressed texture format. 87 A format that divides a texture into blocks that allow the texture to be reduced in size by a fixed ratio; typically 4:1 88 or 6:1. Compressed textures are useful when perfect image/texel fidelity is not necessary but where minimizing 89 memory storage and bandwidth are critical to application performance. 90 91  Extent 92 A vector of integers that describes lengths of N-dimensional array-like objects. 93 94  Global memory 95 On a GPU, global memory is the main off-chip memory store, 96 Informative: Typcially, on current-generation GPUs, global memory is implemented in DRAM, with access times of 97 400-1000 cycles; the GPU clock speed is around 1 Ghz; and may or may not be cached. Global memory is accessed C++ AMP : Language and Programming Model : Version 0.9 : January 2012 Page 3 98 in a coalesced pattern with a granularity of 128 bytes, so when accessing 4 bytes of global memory, 32 successive 99 threads need to read the 32 successive 4-byte addresses, to be fully coalesced. 100 101 Informative: The memory space of current GPUs is typically disjoint from its host system. 102 103  GPGPU: General Purpose computation on Graphics Processing Units, which is a GPU capable of running non- 104 graphics computations. 105 106  GPU: A specialized (co)processor that offloads graphics computation and rendering from the host. As GPUs have 107 evolved, they have become increasingly able to offload non-graphics computations as well (see GPGPU). 108 109  Heterogenous programming 110 A workload that combines kernels executing on data-parallel compute nodes with algorithms running on CPUs. 111 112  Host 113 The operating system proecess and the CPU(s) that it is running on. 114 115  Host thread 116 The operating system thread and the CPU(s) that it is running on. A host thread may initiate a copy operation or 117 parallel loop operation that may run on an accelerator. 118 119  Index 120 A vector of integers that describes an N-dimentional point in iteration space or index space. 121 122  Kernel; Kernel function 123 A program designed to be executed at a C++ AMP call-site. More generally, a kernel is a unit of computation that 124 executes on an accelerator. A kernel function is a special case; it is the root of a logical call graph of functions that 125 execute on an accelerator. A C++ analogy is that it is the “main()” function for an accelerator program 126 127  Perfect loop nest 128 A loop nest in which the body of each outer loop consists of a single statement that is a loop. 129 130  Pixel 131 A pixel, or picture element, represents a single element in a digital image. Typically pixels are composed of multiple 132 color components such as a red, green and blue values. Other color representation exist, including single channel 133 images that just represent intensity or black and white values. 134 135  Reference counting 136 Reference counting is a memory management technique to manage an object’s lifetime. References to an object 137 are counted and the object is kept alive as long as there is at least one reference to it. A reference counted object 138 is destroyed when the last reference disappears. 139 140  SIMD unit 141 Single Instruction Multiple Data. A machine programming model where a single instruction operates over multiple 142 pieces of data. Translating a program to use SIMD is known as vectorization. GPUs have multiple SIMD units, 143 which are the streaming multiprocessors. 144 Informative: An SSE (Nehalem, Phenom) or AVX (Sandy Bridge) or LRBni (Larrabee) vector unit is a SIMD unit or 145 vector processor. 146 147  SMP 148 Symmetric Multi-Processor – standard PC multiprocessor architecure. C++ AMP : Language and Programming Model : Version 0.9 : January 2012