vk_mem_alloc.h source code [Godot/thirdparty/vulkan/vk_mem_alloc.h]

1	//
2	// Copyright (c) 2017-2022 Advanced Micro Devices, Inc. All rights reserved.
3	//
4	// Permission is hereby granted, free of charge, to any person obtaining a copy
5	// of this software and associated documentation files (the "Software"), to deal
6	// in the Software without restriction, including without limitation the rights
7	// to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
8	// copies of the Software, and to permit persons to whom the Software is
9	// furnished to do so, subject to the following conditions:
10	//
11	// The above copyright notice and this permission notice shall be included in
12	// all copies or substantial portions of the Software.
13	//
14	// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
15	// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
16	// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
17	// AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
18	// LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
19	// OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
20	// THE SOFTWARE.
21	//
22
23	#ifndef AMD_VULKAN_MEMORY_ALLOCATOR_H
24	#define AMD_VULKAN_MEMORY_ALLOCATOR_H
25
26	/* \mainpage Vulkan Memory Allocator*
27
28	<b>Version 3.0.1 (2022-05-26)</b>
29
30	Copyright (c) 2017-2022 Advanced Micro Devices, Inc. All rights reserved. \n
31	License: MIT
32
33	<b>API documentation divided into groups:</b> [Modules](modules.html)
34
35	\section main_table_of_contents Table of contents
36
37	- <b>User guide</b>
38	- \subpage quick_start
39	- [Project setup](@ref quick_start_project_setup)
40	- [Initialization](@ref quick_start_initialization)
41	- [Resource allocation](@ref quick_start_resource_allocation)
42	- \subpage choosing_memory_type
43	- [Usage](@ref choosing_memory_type_usage)
44	- [Required and preferred flags](@ref choosing_memory_type_required_preferred_flags)
45	- [Explicit memory types](@ref choosing_memory_type_explicit_memory_types)
46	- [Custom memory pools](@ref choosing_memory_type_custom_memory_pools)
47	- [Dedicated allocations](@ref choosing_memory_type_dedicated_allocations)
48	- \subpage memory_mapping
49	- [Mapping functions](@ref memory_mapping_mapping_functions)
50	- [Persistently mapped memory](@ref memory_mapping_persistently_mapped_memory)
51	- [Cache flush and invalidate](@ref memory_mapping_cache_control)
52	- \subpage staying_within_budget
53	- [Querying for budget](@ref staying_within_budget_querying_for_budget)
54	- [Controlling memory usage](@ref staying_within_budget_controlling_memory_usage)
55	- \subpage resource_aliasing
56	- \subpage custom_memory_pools
57	- [Choosing memory type index](@ref custom_memory_pools_MemTypeIndex)
58	- [Linear allocation algorithm](@ref linear_algorithm)
59	- [Free-at-once](@ref linear_algorithm_free_at_once)
60	- [Stack](@ref linear_algorithm_stack)
61	- [Double stack](@ref linear_algorithm_double_stack)
62	- [Ring buffer](@ref linear_algorithm_ring_buffer)
63	- \subpage defragmentation
64	- \subpage statistics
65	- [Numeric statistics](@ref statistics_numeric_statistics)
66	- [JSON dump](@ref statistics_json_dump)
67	- \subpage allocation_annotation
68	- [Allocation user data](@ref allocation_user_data)
69	- [Allocation names](@ref allocation_names)
70	- \subpage virtual_allocator
71	- \subpage debugging_memory_usage
72	- [Memory initialization](@ref debugging_memory_usage_initialization)
73	- [Margins](@ref debugging_memory_usage_margins)
74	- [Corruption detection](@ref debugging_memory_usage_corruption_detection)
75	- \subpage opengl_interop
76	- \subpage usage_patterns
77	- [GPU-only resource](@ref usage_patterns_gpu_only)
78	- [Staging copy for upload](@ref usage_patterns_staging_copy_upload)
79	- [Readback](@ref usage_patterns_readback)
80	- [Advanced data uploading](@ref usage_patterns_advanced_data_uploading)
81	- [Other use cases](@ref usage_patterns_other_use_cases)
82	- \subpage configuration
83	- [Pointers to Vulkan functions](@ref config_Vulkan_functions)
84	- [Custom host memory allocator](@ref custom_memory_allocator)
85	- [Device memory allocation callbacks](@ref allocation_callbacks)
86	- [Device heap memory limit](@ref heap_memory_limit)
87	- <b>Extension support</b>
88	- \subpage vk_khr_dedicated_allocation
89	- \subpage enabling_buffer_device_address
90	- \subpage vk_ext_memory_priority
91	- \subpage vk_amd_device_coherent_memory
92	- \subpage general_considerations
93	- [Thread safety](@ref general_considerations_thread_safety)
94	- [Versioning and compatibility](@ref general_considerations_versioning_and_compatibility)
95	- [Validation layer warnings](@ref general_considerations_validation_layer_warnings)
96	- [Allocation algorithm](@ref general_considerations_allocation_algorithm)
97	- [Features not supported](@ref general_considerations_features_not_supported)
98
99	\section main_see_also See also
100
101	- [Product page on GPUOpen](https://gpuopen.com/gaming-product/vulkan-memory-allocator/)
102	- [Source repository on GitHub](https://github.com/GPUOpen-LibrariesAndSDKs/VulkanMemoryAllocator)
103
104	\defgroup group_init Library initialization
105
106	\brief API elements related to the initialization and management of the entire library, especially #VmaAllocator object.
107
108	\defgroup group_alloc Memory allocation
109
110	\brief API elements related to the allocation, deallocation, and management of Vulkan memory, buffers, images.
111	Most basic ones being: vmaCreateBuffer(), vmaCreateImage().
112
113	\defgroup group_virtual Virtual allocator
114
115	\brief API elements related to the mechanism of \ref virtual_allocator - using the core allocation algorithm
116	for user-defined purpose without allocating any real GPU memory.
117
118	\defgroup group_stats Statistics
119
120	\brief API elements that query current status of the allocator, from memory usage, budget, to full dump of the internal state in JSON format.
121	See documentation chapter: \ref statistics.
122	*/
123
124
125	#ifdef __cplusplus
126	extern "C" {
127	#endif
128
129	#ifndef VULKAN_H_
130	#ifdef USE_VOLK
131	#include <volk.h>
132	#else
133	#include <vulkan/vulkan.h>
134	#endif
135	#endif
136
137	// Define this macro to declare maximum supported Vulkan version in format AAABBBCCC,
138	// where AAA = major, BBB = minor, CCC = patch.
139	// If you want to use version > 1.0, it still needs to be enabled via VmaAllocatorCreateInfo::vulkanApiVersion.
140	#if !defined(VMA_VULKAN_VERSION)
141	#if defined(VK_VERSION_1_3)
142	#define VMA_VULKAN_VERSION 1003000
143	#elif defined(VK_VERSION_1_2)
144	#define VMA_VULKAN_VERSION 1002000
145	#elif defined(VK_VERSION_1_1)
146	#define VMA_VULKAN_VERSION 1001000
147	#else
148	#define VMA_VULKAN_VERSION 1000000
149	#endif
150	#endif
151
152	#if defined(__ANDROID__) && defined(VK_NO_PROTOTYPES) && VMA_STATIC_VULKAN_FUNCTIONS
153	extern PFN_vkGetInstanceProcAddr vkGetInstanceProcAddr;
154	extern PFN_vkGetDeviceProcAddr vkGetDeviceProcAddr;
155	extern PFN_vkGetPhysicalDeviceProperties vkGetPhysicalDeviceProperties;
156	extern PFN_vkGetPhysicalDeviceMemoryProperties vkGetPhysicalDeviceMemoryProperties;
157	extern PFN_vkAllocateMemory vkAllocateMemory;
158	extern PFN_vkFreeMemory vkFreeMemory;
159	extern PFN_vkMapMemory vkMapMemory;
160	extern PFN_vkUnmapMemory vkUnmapMemory;
161	extern PFN_vkFlushMappedMemoryRanges vkFlushMappedMemoryRanges;
162	extern PFN_vkInvalidateMappedMemoryRanges vkInvalidateMappedMemoryRanges;
163	extern PFN_vkBindBufferMemory vkBindBufferMemory;
164	extern PFN_vkBindImageMemory vkBindImageMemory;
165	extern PFN_vkGetBufferMemoryRequirements vkGetBufferMemoryRequirements;
166	extern PFN_vkGetImageMemoryRequirements vkGetImageMemoryRequirements;
167	extern PFN_vkCreateBuffer vkCreateBuffer;
168	extern PFN_vkDestroyBuffer vkDestroyBuffer;
169	extern PFN_vkCreateImage vkCreateImage;
170	extern PFN_vkDestroyImage vkDestroyImage;
171	extern PFN_vkCmdCopyBuffer vkCmdCopyBuffer;
172	#if VMA_VULKAN_VERSION >= 1001000
173	extern PFN_vkGetBufferMemoryRequirements2 vkGetBufferMemoryRequirements2;
174	extern PFN_vkGetImageMemoryRequirements2 vkGetImageMemoryRequirements2;
175	extern PFN_vkBindBufferMemory2 vkBindBufferMemory2;
176	extern PFN_vkBindImageMemory2 vkBindImageMemory2;
177	extern PFN_vkGetPhysicalDeviceMemoryProperties2 vkGetPhysicalDeviceMemoryProperties2;
178	#endif // #if VMA_VULKAN_VERSION >= 1001000
179	#endif // #if defined(__ANDROID__) && VMA_STATIC_VULKAN_FUNCTIONS && VK_NO_PROTOTYPES
180
181	#if !defined(VMA_DEDICATED_ALLOCATION)
182	#if VK_KHR_get_memory_requirements2 && VK_KHR_dedicated_allocation
183	#define VMA_DEDICATED_ALLOCATION 1
184	#else
185	#define VMA_DEDICATED_ALLOCATION 0
186	#endif
187	#endif
188
189	#if !defined(VMA_BIND_MEMORY2)
190	#if VK_KHR_bind_memory2
191	#define VMA_BIND_MEMORY2 1
192	#else
193	#define VMA_BIND_MEMORY2 0
194	#endif
195	#endif
196
197	#if !defined(VMA_MEMORY_BUDGET)
198	#if VK_EXT_memory_budget && (VK_KHR_get_physical_device_properties2 \|\| VMA_VULKAN_VERSION >= 1001000)
199	#define VMA_MEMORY_BUDGET 1
200	#else
201	#define VMA_MEMORY_BUDGET 0
202	#endif
203	#endif
204
205	// Defined to 1 when VK_KHR_buffer_device_address device extension or equivalent core Vulkan 1.2 feature is defined in its headers.
206	#if !defined(VMA_BUFFER_DEVICE_ADDRESS)
207	#if VK_KHR_buffer_device_address \|\| VMA_VULKAN_VERSION >= 1002000
208	#define VMA_BUFFER_DEVICE_ADDRESS 1
209	#else
210	#define VMA_BUFFER_DEVICE_ADDRESS 0
211	#endif
212	#endif
213
214	// Defined to 1 when VK_EXT_memory_priority device extension is defined in Vulkan headers.
215	#if !defined(VMA_MEMORY_PRIORITY)
216	#if VK_EXT_memory_priority
217	#define VMA_MEMORY_PRIORITY 1
218	#else
219	#define VMA_MEMORY_PRIORITY 0
220	#endif
221	#endif
222
223	// Defined to 1 when VK_KHR_external_memory device extension is defined in Vulkan headers.
224	#if !defined(VMA_EXTERNAL_MEMORY)
225	#if VK_KHR_external_memory
226	#define VMA_EXTERNAL_MEMORY 1
227	#else
228	#define VMA_EXTERNAL_MEMORY 0
229	#endif
230	#endif
231
232	// Define these macros to decorate all public functions with additional code,
233	// before and after returned type, appropriately. This may be useful for
234	// exporting the functions when compiling VMA as a separate library. Example:
235	// #define VMA_CALL_PRE __declspec(dllexport)
236	// #define VMA_CALL_POST __cdecl
237	#ifndef VMA_CALL_PRE
238	#define VMA_CALL_PRE
239	#endif
240	#ifndef VMA_CALL_POST
241	#define VMA_CALL_POST
242	#endif
243
244	// Define this macro to decorate pointers with an attribute specifying the
245	// length of the array they point to if they are not null.
246	//
247	// The length may be one of
248	// - The name of another parameter in the argument list where the pointer is declared
249	// - The name of another member in the struct where the pointer is declared
250	// - The name of a member of a struct type, meaning the value of that member in
251	// the context of the call. For example
252	// VMA_LEN_IF_NOT_NULL("VkPhysicalDeviceMemoryProperties::memoryHeapCount"),
253	// this means the number of memory heaps available in the device associated
254	// with the VmaAllocator being dealt with.
255	#ifndef VMA_LEN_IF_NOT_NULL
256	#define VMA_LEN_IF_NOT_NULL(len)
257	#endif
258
259	// The VMA_NULLABLE macro is defined to be _Nullable when compiling with Clang.
260	// see: https://clang.llvm.org/docs/AttributeReference.html#nullable
261	#ifndef VMA_NULLABLE
262	#ifdef __clang__
263	#define VMA_NULLABLE _Nullable
264	#else
265	#define VMA_NULLABLE
266	#endif
267	#endif
268
269	// The VMA_NOT_NULL macro is defined to be _Nonnull when compiling with Clang.
270	// see: https://clang.llvm.org/docs/AttributeReference.html#nonnull
271	#ifndef VMA_NOT_NULL
272	#ifdef __clang__
273	#define VMA_NOT_NULL _Nonnull
274	#else
275	#define VMA_NOT_NULL
276	#endif
277	#endif
278
279	// If non-dispatchable handles are represented as pointers then we can give
280	// then nullability annotations
281	#ifndef VMA_NOT_NULL_NON_DISPATCHABLE
282	#if defined(__LP64__) \|\| defined(_WIN64) \|\| (defined(__x86_64__) && !defined(__ILP32__) ) \|\| defined(_M_X64) \|\| defined(__ia64) \|\| defined (_M_IA64) \|\| defined(__aarch64__) \|\| defined(__powerpc64__)
283	#define VMA_NOT_NULL_NON_DISPATCHABLE VMA_NOT_NULL
284	#else
285	#define VMA_NOT_NULL_NON_DISPATCHABLE
286	#endif
287	#endif
288
289	#ifndef VMA_NULLABLE_NON_DISPATCHABLE
290	#if defined(__LP64__) \|\| defined(_WIN64) \|\| (defined(__x86_64__) && !defined(__ILP32__) ) \|\| defined(_M_X64) \|\| defined(__ia64) \|\| defined (_M_IA64) \|\| defined(__aarch64__) \|\| defined(__powerpc64__)
291	#define VMA_NULLABLE_NON_DISPATCHABLE VMA_NULLABLE
292	#else
293	#define VMA_NULLABLE_NON_DISPATCHABLE
294	#endif
295	#endif
296
297	#ifndef VMA_STATS_STRING_ENABLED
298	#define VMA_STATS_STRING_ENABLED 1
299	#endif
300
301	////////////////////////////////////////////////////////////////////////////////
302	////////////////////////////////////////////////////////////////////////////////
303	//
304	// INTERFACE
305	//
306	////////////////////////////////////////////////////////////////////////////////
307	////////////////////////////////////////////////////////////////////////////////
308
309	// Sections for managing code placement in file, only for development purposes e.g. for convenient folding inside an IDE.
310	#ifndef _VMA_ENUM_DECLARATIONS
311
312	/**
313	\addtogroup group_init
314	@{
315	*/
316
317	/// Flags for created #VmaAllocator.
318	typedef enum VmaAllocatorCreateFlagBits
319	{
320	/* \brief Allocator and all objects created from it will not be synchronized internally, so you must guarantee they are used from only one thread at a time or synchronized externally by you.*
321
322	Using this flag may increase performance because internal mutexes are not used.
323	*/
324	VMA_ALLOCATOR_CREATE_EXTERNALLY_SYNCHRONIZED_BIT = `0x00000001`,
325	/* \brief Enables usage of VK_KHR_dedicated_allocation extension.*
326
327	The flag works only if VmaAllocatorCreateInfo::vulkanApiVersion `== VK_API_VERSION_1_0`.
328	When it is `VK_API_VERSION_1_1`, the flag is ignored because the extension has been promoted to Vulkan 1.1.
329
330	Using this extension will automatically allocate dedicated blocks of memory for
331	some buffers and images instead of suballocating place for them out of bigger
332	memory blocks (as if you explicitly used #VMA_ALLOCATION_CREATE_DEDICATED_MEMORY_BIT
333	flag) when it is recommended by the driver. It may improve performance on some
334	GPUs.
335
336	You may set this flag only if you found out that following device extensions are
337	supported, you enabled them while creating Vulkan device passed as
338	VmaAllocatorCreateInfo::device, and you want them to be used internally by this
339	library:
340
341	- VK_KHR_get_memory_requirements2 (device extension)
342	- VK_KHR_dedicated_allocation (device extension)
343
344	When this flag is set, you can experience following warnings reported by Vulkan
345	validation layer. You can ignore them.
346
347	> vkBindBufferMemory(): Binding memory to buffer 0x2d but vkGetBufferMemoryRequirements() has not been called on that buffer.
348	*/
349	VMA_ALLOCATOR_CREATE_KHR_DEDICATED_ALLOCATION_BIT = `0x00000002`,
350	/**
351	Enables usage of VK_KHR_bind_memory2 extension.
352
353	The flag works only if VmaAllocatorCreateInfo::vulkanApiVersion `== VK_API_VERSION_1_0`.
354	When it is `VK_API_VERSION_1_1`, the flag is ignored because the extension has been promoted to Vulkan 1.1.
355
356	You may set this flag only if you found out that this device extension is supported,
357	you enabled it while creating Vulkan device passed as VmaAllocatorCreateInfo::device,
358	and you want it to be used internally by this library.
359
360	The extension provides functions `vkBindBufferMemory2KHR` and `vkBindImageMemory2KHR`,
361	which allow to pass a chain of `pNext` structures while binding.
362	This flag is required if you use `pNext` parameter in vmaBindBufferMemory2() or vmaBindImageMemory2().
363	*/
364	VMA_ALLOCATOR_CREATE_KHR_BIND_MEMORY2_BIT = `0x00000004`,
365	/**
366	Enables usage of VK_EXT_memory_budget extension.
367
368	You may set this flag only if you found out that this device extension is supported,
369	you enabled it while creating Vulkan device passed as VmaAllocatorCreateInfo::device,
370	and you want it to be used internally by this library, along with another instance extension
371	VK_KHR_get_physical_device_properties2, which is required by it (or Vulkan 1.1, where this extension is promoted).
372
373	The extension provides query for current memory usage and budget, which will probably
374	be more accurate than an estimation used by the library otherwise.
375	*/
376	VMA_ALLOCATOR_CREATE_EXT_MEMORY_BUDGET_BIT = `0x00000008`,
377	/**
378	Enables usage of VK_AMD_device_coherent_memory extension.
379
380	You may set this flag only if you:
381
382	- found out that this device extension is supported and enabled it while creating Vulkan device passed as VmaAllocatorCreateInfo::device,
383	- checked that `VkPhysicalDeviceCoherentMemoryFeaturesAMD::deviceCoherentMemory` is true and set it while creating the Vulkan device,
384	- want it to be used internally by this library.
385
386	The extension and accompanying device feature provide access to memory types with
387	`VK_MEMORY_PROPERTY_DEVICE_COHERENT_BIT_AMD` and `VK_MEMORY_PROPERTY_DEVICE_UNCACHED_BIT_AMD` flags.
388	They are useful mostly for writing breadcrumb markers - a common method for debugging GPU crash/hang/TDR.
389
390	When the extension is not enabled, such memory types are still enumerated, but their usage is illegal.
391	To protect from this error, if you don't create the allocator with this flag, it will refuse to allocate any memory or create a custom pool in such memory type,
392	returning `VK_ERROR_FEATURE_NOT_PRESENT`.
393	*/
394	VMA_ALLOCATOR_CREATE_AMD_DEVICE_COHERENT_MEMORY_BIT = `0x00000010`,
395	/**
396	Enables usage of "buffer device address" feature, which allows you to use function
397	`vkGetBufferDeviceAddress` to get raw GPU pointer to a buffer and pass it for usage inside a shader.*
398
399	You may set this flag only if you:
400
401	1. (For Vulkan version < 1.2) Found as available and enabled device extension
402	VK_KHR_buffer_device_address.
403	This extension is promoted to core Vulkan 1.2.
404	2. Found as available and enabled device feature `VkPhysicalDeviceBufferDeviceAddressFeatures::bufferDeviceAddress`.
405
406	When this flag is set, you can create buffers with `VK_BUFFER_USAGE_SHADER_DEVICE_ADDRESS_BIT` using VMA.
407	The library automatically adds `VK_MEMORY_ALLOCATE_DEVICE_ADDRESS_BIT` to
408	allocated memory blocks wherever it might be needed.
409
410	For more information, see documentation chapter \ref enabling_buffer_device_address.
411	*/
412	VMA_ALLOCATOR_CREATE_BUFFER_DEVICE_ADDRESS_BIT = `0x00000020`,
413	/**
414	Enables usage of VK_EXT_memory_priority extension in the library.
415
416	You may set this flag only if you found available and enabled this device extension,
417	along with `VkPhysicalDeviceMemoryPriorityFeaturesEXT::memoryPriority == VK_TRUE`,
418	while creating Vulkan device passed as VmaAllocatorCreateInfo::device.
419
420	When this flag is used, VmaAllocationCreateInfo::priority and VmaPoolCreateInfo::priority
421	are used to set priorities of allocated Vulkan memory. Without it, these variables are ignored.
422
423	A priority must be a floating-point value between 0 and 1, indicating the priority of the allocation relative to other memory allocations.
424	Larger values are higher priority. The granularity of the priorities is implementation-dependent.
425	It is automatically passed to every call to `vkAllocateMemory` done by the library using structure `VkMemoryPriorityAllocateInfoEXT`.
426	The value to be used for default priority is 0.5.
427	For more details, see the documentation of the VK_EXT_memory_priority extension.
428	*/
429	VMA_ALLOCATOR_CREATE_EXT_MEMORY_PRIORITY_BIT = `0x00000040`,
430
431	VMA_ALLOCATOR_CREATE_FLAG_BITS_MAX_ENUM = `0x7FFFFFFF`
432	} VmaAllocatorCreateFlagBits;
433	/// See #VmaAllocatorCreateFlagBits.
434	typedef VkFlags VmaAllocatorCreateFlags;
435
436	/* @} /
437
438	/**
439	\addtogroup group_alloc
440	@{
441	*/
442
443	/// \brief Intended usage of the allocated memory.
444	typedef enum VmaMemoryUsage
445	{
446	/* No intended memory usage specified.*
447	Use other members of VmaAllocationCreateInfo to specify your requirements.
448	*/
449	VMA_MEMORY_USAGE_UNKNOWN = `0`,
450	/**
451	\deprecated Obsolete, preserved for backward compatibility.
452	Prefers `VK_MEMORY_PROPERTY_DEVICE_LOCAL_BIT`.
453	*/
454	VMA_MEMORY_USAGE_GPU_ONLY = `1`,
455	/**
456	\deprecated Obsolete, preserved for backward compatibility.
457	Guarantees `VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT` and `VK_MEMORY_PROPERTY_HOST_COHERENT_BIT`.
458	*/
459	VMA_MEMORY_USAGE_CPU_ONLY = `2`,
460	/**
461	\deprecated Obsolete, preserved for backward compatibility.
462	Guarantees `VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT`, prefers `VK_MEMORY_PROPERTY_DEVICE_LOCAL_BIT`.
463	*/
464	VMA_MEMORY_USAGE_CPU_TO_GPU = `3`,
465	/**
466	\deprecated Obsolete, preserved for backward compatibility.
467	Guarantees `VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT`, prefers `VK_MEMORY_PROPERTY_HOST_CACHED_BIT`.
468	*/
469	VMA_MEMORY_USAGE_GPU_TO_CPU = `4`,
470	/**
471	\deprecated Obsolete, preserved for backward compatibility.
472	Prefers not `VK_MEMORY_PROPERTY_DEVICE_LOCAL_BIT`.
473	*/
474	VMA_MEMORY_USAGE_CPU_COPY = `5`,
475	/**
476	Lazily allocated GPU memory having `VK_MEMORY_PROPERTY_LAZILY_ALLOCATED_BIT`.
477	Exists mostly on mobile platforms. Using it on desktop PC or other GPUs with no such memory type present will fail the allocation.
478
479	Usage: Memory for transient attachment images (color attachments, depth attachments etc.), created with `VK_IMAGE_USAGE_TRANSIENT_ATTACHMENT_BIT`.
480
481	Allocations with this usage are always created as dedicated - it implies #VMA_ALLOCATION_CREATE_DEDICATED_MEMORY_BIT.
482	*/
483	VMA_MEMORY_USAGE_GPU_LAZILY_ALLOCATED = `6`,
484	/**
485	Selects best memory type automatically.
486	This flag is recommended for most common use cases.
487
488	When using this flag, if you want to map the allocation (using vmaMapMemory() or #VMA_ALLOCATION_CREATE_MAPPED_BIT),
489	you must pass one of the flags: #VMA_ALLOCATION_CREATE_HOST_ACCESS_SEQUENTIAL_WRITE_BIT or #VMA_ALLOCATION_CREATE_HOST_ACCESS_RANDOM_BIT
490	in VmaAllocationCreateInfo::flags.
491
492	It can be used only with functions that let the library know `VkBufferCreateInfo` or `VkImageCreateInfo`, e.g.
493	vmaCreateBuffer(), vmaCreateImage(), vmaFindMemoryTypeIndexForBufferInfo(), vmaFindMemoryTypeIndexForImageInfo()
494	and not with generic memory allocation functions.
495	*/
496	VMA_MEMORY_USAGE_AUTO = `7`,
497	/**
498	Selects best memory type automatically with preference for GPU (device) memory.
499
500	When using this flag, if you want to map the allocation (using vmaMapMemory() or #VMA_ALLOCATION_CREATE_MAPPED_BIT),
501	you must pass one of the flags: #VMA_ALLOCATION_CREATE_HOST_ACCESS_SEQUENTIAL_WRITE_BIT or #VMA_ALLOCATION_CREATE_HOST_ACCESS_RANDOM_BIT
502	in VmaAllocationCreateInfo::flags.
503
504	It can be used only with functions that let the library know `VkBufferCreateInfo` or `VkImageCreateInfo`, e.g.
505	vmaCreateBuffer(), vmaCreateImage(), vmaFindMemoryTypeIndexForBufferInfo(), vmaFindMemoryTypeIndexForImageInfo()
506	and not with generic memory allocation functions.
507	*/
508	VMA_MEMORY_USAGE_AUTO_PREFER_DEVICE = `8`,
509	/**
510	Selects best memory type automatically with preference for CPU (host) memory.
511
512	When using this flag, if you want to map the allocation (using vmaMapMemory() or #VMA_ALLOCATION_CREATE_MAPPED_BIT),
513	you must pass one of the flags: #VMA_ALLOCATION_CREATE_HOST_ACCESS_SEQUENTIAL_WRITE_BIT or #VMA_ALLOCATION_CREATE_HOST_ACCESS_RANDOM_BIT
514	in VmaAllocationCreateInfo::flags.
515
516	It can be used only with functions that let the library know `VkBufferCreateInfo` or `VkImageCreateInfo`, e.g.
517	vmaCreateBuffer(), vmaCreateImage(), vmaFindMemoryTypeIndexForBufferInfo(), vmaFindMemoryTypeIndexForImageInfo()
518	and not with generic memory allocation functions.
519	*/
520	VMA_MEMORY_USAGE_AUTO_PREFER_HOST = `9`,
521
522	VMA_MEMORY_USAGE_MAX_ENUM = `0x7FFFFFFF`
523	} VmaMemoryUsage;
524
525	/// Flags to be passed as VmaAllocationCreateInfo::flags.
526	typedef enum VmaAllocationCreateFlagBits
527	{
528	/* \brief Set this flag if the allocation should have its own memory block.*
529
530	Use it for special, big resources, like fullscreen images used as attachments.
531	*/
532	VMA_ALLOCATION_CREATE_DEDICATED_MEMORY_BIT = `0x00000001`,
533
534	/* \brief Set this flag to only try to allocate from existing `VkDeviceMemory` blocks and never create new such block.*
535
536	If new allocation cannot be placed in any of the existing blocks, allocation
537	fails with `VK_ERROR_OUT_OF_DEVICE_MEMORY` error.
538
539	You should not use #VMA_ALLOCATION_CREATE_DEDICATED_MEMORY_BIT and
540	#VMA_ALLOCATION_CREATE_NEVER_ALLOCATE_BIT at the same time. It makes no sense.
541	*/
542	VMA_ALLOCATION_CREATE_NEVER_ALLOCATE_BIT = `0x00000002`,
543	/* \brief Set this flag to use a memory that will be persistently mapped and retrieve pointer to it.*
544
545	Pointer to mapped memory will be returned through VmaAllocationInfo::pMappedData.
546
547	It is valid to use this flag for allocation made from memory type that is not
548	`HOST_VISIBLE`. This flag is then ignored and memory is not mapped. This is
549	useful if you need an allocation that is efficient to use on GPU
550	(`DEVICE_LOCAL`) and still want to map it directly if possible on platforms that
551	support it (e.g. Intel GPU).
552	*/
553	VMA_ALLOCATION_CREATE_MAPPED_BIT = `0x00000004`,
554	/* \deprecated Preserved for backward compatibility. Consider using vmaSetAllocationName() instead.*
555
556	Set this flag to treat VmaAllocationCreateInfo::pUserData as pointer to a
557	null-terminated string. Instead of copying pointer value, a local copy of the
558	string is made and stored in allocation's `pName`. The string is automatically
559	freed together with the allocation. It is also used in vmaBuildStatsString().
560	*/
561	VMA_ALLOCATION_CREATE_USER_DATA_COPY_STRING_BIT = `0x00000020`,
562	/* Allocation will be created from upper stack in a double stack pool.*
563
564	This flag is only allowed for custom pools created with #VMA_POOL_CREATE_LINEAR_ALGORITHM_BIT flag.
565	*/
566	VMA_ALLOCATION_CREATE_UPPER_ADDRESS_BIT = `0x00000040`,
567	/* Create both buffer/image and allocation, but don't bind them together.*
568	It is useful when you want to bind yourself to do some more advanced binding, e.g. using some extensions.
569	The flag is meaningful only with functions that bind by default: vmaCreateBuffer(), vmaCreateImage().
570	Otherwise it is ignored.
571
572	If you want to make sure the new buffer/image is not tied to the new memory allocation
573	through `VkMemoryDedicatedAllocateInfoKHR` structure in case the allocation ends up in its own memory block,
574	use also flag #VMA_ALLOCATION_CREATE_CAN_ALIAS_BIT.
575	*/
576	VMA_ALLOCATION_CREATE_DONT_BIND_BIT = `0x00000080`,
577	/* Create allocation only if additional device memory required for it, if any, won't exceed*
578	memory budget. Otherwise return `VK_ERROR_OUT_OF_DEVICE_MEMORY`.
579	*/
580	VMA_ALLOCATION_CREATE_WITHIN_BUDGET_BIT = `0x00000100`,
581	/* \brief Set this flag if the allocated memory will have aliasing resources.*
582
583	Usage of this flag prevents supplying `VkMemoryDedicatedAllocateInfoKHR` when #VMA_ALLOCATION_CREATE_DEDICATED_MEMORY_BIT is specified.
584	Otherwise created dedicated memory will not be suitable for aliasing resources, resulting in Vulkan Validation Layer errors.
585	*/
586	VMA_ALLOCATION_CREATE_CAN_ALIAS_BIT = `0x00000200`,
587	/**
588	Requests possibility to map the allocation (using vmaMapMemory() or #VMA_ALLOCATION_CREATE_MAPPED_BIT).
589
590	- If you use #VMA_MEMORY_USAGE_AUTO or other `VMA_MEMORY_USAGE_AUTO` value,*
591	you must use this flag to be able to map the allocation. Otherwise, mapping is incorrect.
592	- If you use other value of #VmaMemoryUsage, this flag is ignored and mapping is always possible in memory types that are `HOST_VISIBLE`.
593	This includes allocations created in \ref custom_memory_pools.
594
595	Declares that mapped memory will only be written sequentially, e.g. using `memcpy()` or a loop writing number-by-number,
596	never read or accessed randomly, so a memory type can be selected that is uncached and write-combined.
597
598	\warning Violating this declaration may work correctly, but will likely be very slow.
599	Watch out for implicit reads introduced by doing e.g. `pMappedData[i] += x;`
600	Better prepare your data in a local variable and `memcpy()` it to the mapped pointer all at once.
601	*/
602	VMA_ALLOCATION_CREATE_HOST_ACCESS_SEQUENTIAL_WRITE_BIT = `0x00000400`,
603	/**
604	Requests possibility to map the allocation (using vmaMapMemory() or #VMA_ALLOCATION_CREATE_MAPPED_BIT).
605
606	- If you use #VMA_MEMORY_USAGE_AUTO or other `VMA_MEMORY_USAGE_AUTO` value,*
607	you must use this flag to be able to map the allocation. Otherwise, mapping is incorrect.
608	- If you use other value of #VmaMemoryUsage, this flag is ignored and mapping is always possible in memory types that are `HOST_VISIBLE`.
609	This includes allocations created in \ref custom_memory_pools.
610
611	Declares that mapped memory can be read, written, and accessed in random order,
612	so a `HOST_CACHED` memory type is required.
613	*/
614	VMA_ALLOCATION_CREATE_HOST_ACCESS_RANDOM_BIT = `0x00000800`,
615	/**
616	Together with #VMA_ALLOCATION_CREATE_HOST_ACCESS_SEQUENTIAL_WRITE_BIT or #VMA_ALLOCATION_CREATE_HOST_ACCESS_RANDOM_BIT,
617	it says that despite request for host access, a not-`HOST_VISIBLE` memory type can be selected
618	if it may improve performance.
619
620	By using this flag, you declare that you will check if the allocation ended up in a `HOST_VISIBLE` memory type
621	(e.g. using vmaGetAllocationMemoryProperties()) and if not, you will create some "staging" buffer and
622	issue an explicit transfer to write/read your data.
623	To prepare for this possibility, don't forget to add appropriate flags like
624	`VK_BUFFER_USAGE_TRANSFER_DST_BIT`, `VK_BUFFER_USAGE_TRANSFER_SRC_BIT` to the parameters of created buffer or image.
625	*/
626	VMA_ALLOCATION_CREATE_HOST_ACCESS_ALLOW_TRANSFER_INSTEAD_BIT = `0x00001000`,
627	/* Allocation strategy that chooses smallest possible free range for the allocation*
628	to minimize memory usage and fragmentation, possibly at the expense of allocation time.
629	*/
630	VMA_ALLOCATION_CREATE_STRATEGY_MIN_MEMORY_BIT = `0x00010000`,
631	/* Allocation strategy that chooses first suitable free range for the allocation -*
632	not necessarily in terms of the smallest offset but the one that is easiest and fastest to find
633	to minimize allocation time, possibly at the expense of allocation quality.
634	*/
635	VMA_ALLOCATION_CREATE_STRATEGY_MIN_TIME_BIT = `0x00020000`,
636	/* Allocation strategy that chooses always the lowest offset in available space.*
637	This is not the most efficient strategy but achieves highly packed data.
638	Used internally by defragmentation, not recomended in typical usage.
639	*/
640	VMA_ALLOCATION_CREATE_STRATEGY_MIN_OFFSET_BIT = `0x00040000`,
641	/* Alias to #VMA_ALLOCATION_CREATE_STRATEGY_MIN_MEMORY_BIT.*
642	*/
643	VMA_ALLOCATION_CREATE_STRATEGY_BEST_FIT_BIT = VMA_ALLOCATION_CREATE_STRATEGY_MIN_MEMORY_BIT,
644	/* Alias to #VMA_ALLOCATION_CREATE_STRATEGY_MIN_TIME_BIT.*
645	*/
646	VMA_ALLOCATION_CREATE_STRATEGY_FIRST_FIT_BIT = VMA_ALLOCATION_CREATE_STRATEGY_MIN_TIME_BIT,
647	/* A bit mask to extract only `STRATEGY` bits from entire set of flags.*
648	*/
649	VMA_ALLOCATION_CREATE_STRATEGY_MASK =
650	VMA_ALLOCATION_CREATE_STRATEGY_MIN_MEMORY_BIT \|
651	VMA_ALLOCATION_CREATE_STRATEGY_MIN_TIME_BIT \|
652	VMA_ALLOCATION_CREATE_STRATEGY_MIN_OFFSET_BIT,
653
654	VMA_ALLOCATION_CREATE_FLAG_BITS_MAX_ENUM = `0x7FFFFFFF`
655	} VmaAllocationCreateFlagBits;
656	/// See #VmaAllocationCreateFlagBits.
657	typedef VkFlags VmaAllocationCreateFlags;
658
659	/// Flags to be passed as VmaPoolCreateInfo::flags.
660	typedef enum VmaPoolCreateFlagBits
661	{
662	/* \brief Use this flag if you always allocate only buffers and linear images or only optimal images out of this pool and so Buffer-Image Granularity can be ignored.*
663
664	This is an optional optimization flag.
665
666	If you always allocate using vmaCreateBuffer(), vmaCreateImage(),
667	vmaAllocateMemoryForBuffer(), then you don't need to use it because allocator
668	knows exact type of your allocations so it can handle Buffer-Image Granularity
669	in the optimal way.
670
671	If you also allocate using vmaAllocateMemoryForImage() or vmaAllocateMemory(),
672	exact type of such allocations is not known, so allocator must be conservative
673	in handling Buffer-Image Granularity, which can lead to suboptimal allocation
674	(wasted memory). In that case, if you can make sure you always allocate only
675	buffers and linear images or only optimal images out of this pool, use this flag
676	to make allocator disregard Buffer-Image Granularity and so make allocations
677	faster and more optimal.
678	*/
679	VMA_POOL_CREATE_IGNORE_BUFFER_IMAGE_GRANULARITY_BIT = `0x00000002`,
680
681	/* \brief Enables alternative, linear allocation algorithm in this pool.*
682
683	Specify this flag to enable linear allocation algorithm, which always creates
684	new allocations after last one and doesn't reuse space from allocations freed in
685	between. It trades memory consumption for simplified algorithm and data
686	structure, which has better performance and uses less memory for metadata.
687
688	By using this flag, you can achieve behavior of free-at-once, stack,
689	ring buffer, and double stack.
690	For details, see documentation chapter \ref linear_algorithm.
691	*/
692	VMA_POOL_CREATE_LINEAR_ALGORITHM_BIT = `0x00000004`,
693
694	/* Bit mask to extract only `ALGORITHM` bits from entire set of flags.*
695	*/
696	VMA_POOL_CREATE_ALGORITHM_MASK =
697	VMA_POOL_CREATE_LINEAR_ALGORITHM_BIT,
698
699	VMA_POOL_CREATE_FLAG_BITS_MAX_ENUM = `0x7FFFFFFF`
700	} VmaPoolCreateFlagBits;
701	/// Flags to be passed as VmaPoolCreateInfo::flags. See #VmaPoolCreateFlagBits.
702	typedef VkFlags VmaPoolCreateFlags;
703
704	/// Flags to be passed as VmaDefragmentationInfo::flags.
705	typedef enum VmaDefragmentationFlagBits
706	{
707	/ \brief Use simple but fast algorithm for defragmentation.*
708	May not achieve best results but will require least time to compute and least allocations to copy.
709	*/
710	VMA_DEFRAGMENTATION_FLAG_ALGORITHM_FAST_BIT = `0x1`,
711	/ \brief Default defragmentation algorithm, applied also when no `ALGORITHM` flag is specified.*
712	Offers a balance between defragmentation quality and the amount of allocations and bytes that need to be moved.
713	*/
714	VMA_DEFRAGMENTATION_FLAG_ALGORITHM_BALANCED_BIT = `0x2`,
715	/ \brief Perform full defragmentation of memory.*
716	Can result in notably more time to compute and allocations to copy, but will achieve best memory packing.
717	*/
718	VMA_DEFRAGMENTATION_FLAG_ALGORITHM_FULL_BIT = `0x4`,
719	/* \brief Use the most roboust algorithm at the cost of time to compute and number of copies to make.*
720	Only available when bufferImageGranularity is greater than 1, since it aims to reduce
721	alignment issues between different types of resources.
722	Otherwise falls back to same behavior as #VMA_DEFRAGMENTATION_FLAG_ALGORITHM_FULL_BIT.
723	*/
724	VMA_DEFRAGMENTATION_FLAG_ALGORITHM_EXTENSIVE_BIT = `0x8`,
725
726	/// A bit mask to extract only `ALGORITHM` bits from entire set of flags.
727	VMA_DEFRAGMENTATION_FLAG_ALGORITHM_MASK =
728	VMA_DEFRAGMENTATION_FLAG_ALGORITHM_FAST_BIT \|
729	VMA_DEFRAGMENTATION_FLAG_ALGORITHM_BALANCED_BIT \|
730	VMA_DEFRAGMENTATION_FLAG_ALGORITHM_FULL_BIT \|
731	VMA_DEFRAGMENTATION_FLAG_ALGORITHM_EXTENSIVE_BIT,
732
733	VMA_DEFRAGMENTATION_FLAG_BITS_MAX_ENUM = `0x7FFFFFFF`
734	} VmaDefragmentationFlagBits;
735	/// See #VmaDefragmentationFlagBits.
736	typedef VkFlags VmaDefragmentationFlags;
737
738	/// Operation performed on single defragmentation move. See structure #VmaDefragmentationMove.
739	typedef enum VmaDefragmentationMoveOperation
740	{
741	/// Buffer/image has been recreated at `dstTmpAllocation`, data has been copied, old buffer/image has been destroyed. `srcAllocation` should be changed to point to the new place. This is the default value set by vmaBeginDefragmentationPass().
742	VMA_DEFRAGMENTATION_MOVE_OPERATION_COPY = `0`,
743	/// Set this value if you cannot move the allocation. New place reserved at `dstTmpAllocation` will be freed. `srcAllocation` will remain unchanged.
744	VMA_DEFRAGMENTATION_MOVE_OPERATION_IGNORE = `1`,
745	/// Set this value if you decide to abandon the allocation and you destroyed the buffer/image. New place reserved at `dstTmpAllocation` will be freed, along with `srcAllocation`, which will be destroyed.
746	VMA_DEFRAGMENTATION_MOVE_OPERATION_DESTROY = `2`,
747	} VmaDefragmentationMoveOperation;
748
749	/* @} /
750
751	/**
752	\addtogroup group_virtual
753	@{
754	*/
755
756	/// Flags to be passed as VmaVirtualBlockCreateInfo::flags.
757	typedef enum VmaVirtualBlockCreateFlagBits
758	{
759	/* \brief Enables alternative, linear allocation algorithm in this virtual block.*
760
761	Specify this flag to enable linear allocation algorithm, which always creates
762	new allocations after last one and doesn't reuse space from allocations freed in
763	between. It trades memory consumption for simplified algorithm and data
764	structure, which has better performance and uses less memory for metadata.
765
766	By using this flag, you can achieve behavior of free-at-once, stack,
767	ring buffer, and double stack.
768	For details, see documentation chapter \ref linear_algorithm.
769	*/
770	VMA_VIRTUAL_BLOCK_CREATE_LINEAR_ALGORITHM_BIT = `0x00000001`,
771
772	/* \brief Bit mask to extract only `ALGORITHM` bits from entire set of flags.*
773	*/
774	VMA_VIRTUAL_BLOCK_CREATE_ALGORITHM_MASK =
775	VMA_VIRTUAL_BLOCK_CREATE_LINEAR_ALGORITHM_BIT,
776
777	VMA_VIRTUAL_BLOCK_CREATE_FLAG_BITS_MAX_ENUM = `0x7FFFFFFF`
778	} VmaVirtualBlockCreateFlagBits;
779	/// Flags to be passed as VmaVirtualBlockCreateInfo::flags. See #VmaVirtualBlockCreateFlagBits.
780	typedef VkFlags VmaVirtualBlockCreateFlags;
781
782	/// Flags to be passed as VmaVirtualAllocationCreateInfo::flags.
783	typedef enum VmaVirtualAllocationCreateFlagBits
784	{
785	/* \brief Allocation will be created from upper stack in a double stack pool.*
786
787	This flag is only allowed for virtual blocks created with #VMA_VIRTUAL_BLOCK_CREATE_LINEAR_ALGORITHM_BIT flag.
788	*/
789	VMA_VIRTUAL_ALLOCATION_CREATE_UPPER_ADDRESS_BIT = VMA_ALLOCATION_CREATE_UPPER_ADDRESS_BIT,
790	/* \brief Allocation strategy that tries to minimize memory usage.*
791	*/
792	VMA_VIRTUAL_ALLOCATION_CREATE_STRATEGY_MIN_MEMORY_BIT = VMA_ALLOCATION_CREATE_STRATEGY_MIN_MEMORY_BIT,
793	/* \brief Allocation strategy that tries to minimize allocation time.*
794	*/
795	VMA_VIRTUAL_ALLOCATION_CREATE_STRATEGY_MIN_TIME_BIT = VMA_ALLOCATION_CREATE_STRATEGY_MIN_TIME_BIT,
796	/* Allocation strategy that chooses always the lowest offset in available space.*
797	This is not the most efficient strategy but achieves highly packed data.
798	*/
799	VMA_VIRTUAL_ALLOCATION_CREATE_STRATEGY_MIN_OFFSET_BIT = VMA_ALLOCATION_CREATE_STRATEGY_MIN_OFFSET_BIT,
800	/* \brief A bit mask to extract only `STRATEGY` bits from entire set of flags.*
801
802	These strategy flags are binary compatible with equivalent flags in #VmaAllocationCreateFlagBits.
803	*/
804	VMA_VIRTUAL_ALLOCATION_CREATE_STRATEGY_MASK = VMA_ALLOCATION_CREATE_STRATEGY_MASK,
805
806	VMA_VIRTUAL_ALLOCATION_CREATE_FLAG_BITS_MAX_ENUM = `0x7FFFFFFF`
807	} VmaVirtualAllocationCreateFlagBits;
808	/// Flags to be passed as VmaVirtualAllocationCreateInfo::flags. See #VmaVirtualAllocationCreateFlagBits.
809	typedef VkFlags VmaVirtualAllocationCreateFlags;
810
811	/* @} /
812
813	#endif // _VMA_ENUM_DECLARATIONS
814
815	#ifndef _VMA_DATA_TYPES_DECLARATIONS
816
817	/**
818	\addtogroup group_init
819	@{ /*
820
821	/* \struct VmaAllocator*
822	\brief Represents main object of this library initialized.
823
824	Fill structure #VmaAllocatorCreateInfo and call function vmaCreateAllocator() to create it.
825	Call function vmaDestroyAllocator() to destroy it.
826
827	It is recommended to create just one object of this type per `VkDevice` object,
828	right after Vulkan is initialized and keep it alive until before Vulkan device is destroyed.
829	*/
830	VK_DEFINE_HANDLE(VmaAllocator)
831
832	/* @} /
833
834	/**
835	\addtogroup group_alloc
836	@{
837	*/
838
839	/* \struct VmaPool*
840	\brief Represents custom memory pool
841
842	Fill structure VmaPoolCreateInfo and call function vmaCreatePool() to create it.
843	Call function vmaDestroyPool() to destroy it.
844
845	For more information see [Custom memory pools](@ref choosing_memory_type_custom_memory_pools).
846	*/
847	VK_DEFINE_HANDLE(VmaPool)
848
849	/* \struct VmaAllocation*
850	\brief Represents single memory allocation.
851
852	It may be either dedicated block of `VkDeviceMemory` or a specific region of a bigger block of this type
853	plus unique offset.
854
855	There are multiple ways to create such object.
856	You need to fill structure VmaAllocationCreateInfo.
857	For more information see [Choosing memory type](@ref choosing_memory_type).
858
859	Although the library provides convenience functions that create Vulkan buffer or image,
860	allocate memory for it and bind them together,
861	binding of the allocation to a buffer or an image is out of scope of the allocation itself.
862	Allocation object can exist without buffer/image bound,
863	binding can be done manually by the user, and destruction of it can be done
864	independently of destruction of the allocation.
865
866	The object also remembers its size and some other information.
867	To retrieve this information, use function vmaGetAllocationInfo() and inspect
868	returned structure VmaAllocationInfo.
869	*/
870	VK_DEFINE_HANDLE(VmaAllocation)
871
872	/* \struct VmaDefragmentationContext*
873	\brief An opaque object that represents started defragmentation process.
874
875	Fill structure #VmaDefragmentationInfo and call function vmaBeginDefragmentation() to create it.
876	Call function vmaEndDefragmentation() to destroy it.
877	*/
878	VK_DEFINE_HANDLE(VmaDefragmentationContext)
879
880	/* @} /
881
882	/**
883	\addtogroup group_virtual
884	@{
885	*/
886
887	/* \struct VmaVirtualAllocation*
888	\brief Represents single memory allocation done inside VmaVirtualBlock.
889
890	Use it as a unique identifier to virtual allocation within the single block.
891
892	Use value `VK_NULL_HANDLE` to represent a null/invalid allocation.
893	*/
894	VK_DEFINE_NON_DISPATCHABLE_HANDLE(VmaVirtualAllocation);
895
896	/* @} /
897
898	/**
899	\addtogroup group_virtual
900	@{
901	*/
902
903	/* \struct VmaVirtualBlock*
904	\brief Handle to a virtual block object that allows to use core allocation algorithm without allocating any real GPU memory.
905
906	Fill in #VmaVirtualBlockCreateInfo structure and use vmaCreateVirtualBlock() to create it. Use vmaDestroyVirtualBlock() to destroy it.
907	For more information, see documentation chapter \ref virtual_allocator.
908
909	This object is not thread-safe - should not be used from multiple threads simultaneously, must be synchronized externally.
910	*/
911	VK_DEFINE_HANDLE(VmaVirtualBlock)
912
913	/* @} /
914
915	/**
916	\addtogroup group_init
917	@{
918	*/
919
920	/// Callback function called after successful vkAllocateMemory.
921	typedef void (VKAPI_PTR* PFN_vmaAllocateDeviceMemoryFunction)(
922	VmaAllocator VMA_NOT_NULL allocator,
923	uint32_t memoryType,
924	VkDeviceMemory VMA_NOT_NULL_NON_DISPATCHABLE memory,
925	VkDeviceSize size,
926	void* VMA_NULLABLE pUserData);
927
928	/// Callback function called before vkFreeMemory.
929	typedef void (VKAPI_PTR* PFN_vmaFreeDeviceMemoryFunction)(
930	VmaAllocator VMA_NOT_NULL allocator,
931	uint32_t memoryType,
932	VkDeviceMemory VMA_NOT_NULL_NON_DISPATCHABLE memory,
933	VkDeviceSize size,
934	void* VMA_NULLABLE pUserData);
935
936	/* \brief Set of callbacks that the library will call for `vkAllocateMemory` and `vkFreeMemory`.*
937
938	Provided for informative purpose, e.g. to gather statistics about number of
939	allocations or total amount of memory allocated in Vulkan.
940
941	Used in VmaAllocatorCreateInfo::pDeviceMemoryCallbacks.
942	*/
943	typedef struct VmaDeviceMemoryCallbacks
944	{
945	/// Optional, can be null.
946	PFN_vmaAllocateDeviceMemoryFunction VMA_NULLABLE pfnAllocate;
947	/// Optional, can be null.
948	PFN_vmaFreeDeviceMemoryFunction VMA_NULLABLE pfnFree;
949	/// Optional, can be null.
950	void* VMA_NULLABLE pUserData;
951	} VmaDeviceMemoryCallbacks;
952
953	/* \brief Pointers to some Vulkan functions - a subset used by the library.*
954
955	Used in VmaAllocatorCreateInfo::pVulkanFunctions.
956	*/
957	typedef struct VmaVulkanFunctions
958	{
959	/// Required when using VMA_DYNAMIC_VULKAN_FUNCTIONS.
960	PFN_vkGetInstanceProcAddr VMA_NULLABLE vkGetInstanceProcAddr;
961	/// Required when using VMA_DYNAMIC_VULKAN_FUNCTIONS.
962	PFN_vkGetDeviceProcAddr VMA_NULLABLE vkGetDeviceProcAddr;
963	PFN_vkGetPhysicalDeviceProperties VMA_NULLABLE vkGetPhysicalDeviceProperties;
964	PFN_vkGetPhysicalDeviceMemoryProperties VMA_NULLABLE vkGetPhysicalDeviceMemoryProperties;
965	PFN_vkAllocateMemory VMA_NULLABLE vkAllocateMemory;
966	PFN_vkFreeMemory VMA_NULLABLE vkFreeMemory;
967	PFN_vkMapMemory VMA_NULLABLE vkMapMemory;
968	PFN_vkUnmapMemory VMA_NULLABLE vkUnmapMemory;
969	PFN_vkFlushMappedMemoryRanges VMA_NULLABLE vkFlushMappedMemoryRanges;
970	PFN_vkInvalidateMappedMemoryRanges VMA_NULLABLE vkInvalidateMappedMemoryRanges;
971	PFN_vkBindBufferMemory VMA_NULLABLE vkBindBufferMemory;
972	PFN_vkBindImageMemory VMA_NULLABLE vkBindImageMemory;
973	PFN_vkGetBufferMemoryRequirements VMA_NULLABLE vkGetBufferMemoryRequirements;
974	PFN_vkGetImageMemoryRequirements VMA_NULLABLE vkGetImageMemoryRequirements;
975	PFN_vkCreateBuffer VMA_NULLABLE vkCreateBuffer;
976	PFN_vkDestroyBuffer VMA_NULLABLE vkDestroyBuffer;
977	PFN_vkCreateImage VMA_NULLABLE vkCreateImage;
978	PFN_vkDestroyImage VMA_NULLABLE vkDestroyImage;
979	PFN_vkCmdCopyBuffer VMA_NULLABLE vkCmdCopyBuffer;
980	#if VMA_DEDICATED_ALLOCATION \|\| VMA_VULKAN_VERSION >= 1001000
981	/// Fetch "vkGetBufferMemoryRequirements2" on Vulkan >= 1.1, fetch "vkGetBufferMemoryRequirements2KHR" when using VK_KHR_dedicated_allocation extension.
982	PFN_vkGetBufferMemoryRequirements2KHR VMA_NULLABLE vkGetBufferMemoryRequirements2KHR;
983	/// Fetch "vkGetImageMemoryRequirements2" on Vulkan >= 1.1, fetch "vkGetImageMemoryRequirements2KHR" when using VK_KHR_dedicated_allocation extension.
984	PFN_vkGetImageMemoryRequirements2KHR VMA_NULLABLE vkGetImageMemoryRequirements2KHR;
985	#endif
986	#if VMA_BIND_MEMORY2 \|\| VMA_VULKAN_VERSION >= 1001000
987	/// Fetch "vkBindBufferMemory2" on Vulkan >= 1.1, fetch "vkBindBufferMemory2KHR" when using VK_KHR_bind_memory2 extension.
988	PFN_vkBindBufferMemory2KHR VMA_NULLABLE vkBindBufferMemory2KHR;
989	/// Fetch "vkBindImageMemory2" on Vulkan >= 1.1, fetch "vkBindImageMemory2KHR" when using VK_KHR_bind_memory2 extension.
990	PFN_vkBindImageMemory2KHR VMA_NULLABLE vkBindImageMemory2KHR;
991	#endif
992	#if VMA_MEMORY_BUDGET \|\| VMA_VULKAN_VERSION >= 1001000
993	PFN_vkGetPhysicalDeviceMemoryProperties2KHR VMA_NULLABLE vkGetPhysicalDeviceMemoryProperties2KHR;
994	#endif
995	#if VMA_VULKAN_VERSION >= 1003000
996	/// Fetch from "vkGetDeviceBufferMemoryRequirements" on Vulkan >= 1.3, but you can also fetch it from "vkGetDeviceBufferMemoryRequirementsKHR" if you enabled extension VK_KHR_maintenance4.
997	PFN_vkGetDeviceBufferMemoryRequirements VMA_NULLABLE vkGetDeviceBufferMemoryRequirements;
998	/// Fetch from "vkGetDeviceImageMemoryRequirements" on Vulkan >= 1.3, but you can also fetch it from "vkGetDeviceImageMemoryRequirementsKHR" if you enabled extension VK_KHR_maintenance4.
999	PFN_vkGetDeviceImageMemoryRequirements VMA_NULLABLE vkGetDeviceImageMemoryRequirements;
1000	#endif
1001	} VmaVulkanFunctions;
1002
1003	/// Description of a Allocator to be created.
1004	typedef struct VmaAllocatorCreateInfo
1005	{
1006	/// Flags for created allocator. Use #VmaAllocatorCreateFlagBits enum.
1007	VmaAllocatorCreateFlags flags;
1008	/// Vulkan physical device.
1009	/* It must be valid throughout whole lifetime of created allocator. /
1010	VkPhysicalDevice VMA_NOT_NULL physicalDevice;
1011	/// Vulkan device.
1012	/* It must be valid throughout whole lifetime of created allocator. /
1013	VkDevice VMA_NOT_NULL device;
1014	/// Preferred size of a single `VkDeviceMemory` block to be allocated from large heaps > 1 GiB. Optional.
1015	/* Set to 0 to use default, which is currently 256 MiB. /
1016	VkDeviceSize preferredLargeHeapBlockSize;
1017	/// Custom CPU memory allocation callbacks. Optional.
1018	/* Optional, can be null. When specified, will also be used for all CPU-side memory allocations. /
1019	const VkAllocationCallbacks* VMA_NULLABLE pAllocationCallbacks;
1020	/// Informative callbacks for `vkAllocateMemory`, `vkFreeMemory`. Optional.
1021	/* Optional, can be null. /
1022	const VmaDeviceMemoryCallbacks* VMA_NULLABLE pDeviceMemoryCallbacks;
1023	/* \brief Either null or a pointer to an array of limits on maximum number of bytes that can be allocated out of particular Vulkan memory heap.*
1024
1025	If not NULL, it must be a pointer to an array of
1026	`VkPhysicalDeviceMemoryProperties::memoryHeapCount` elements, defining limit on
1027	maximum number of bytes that can be allocated out of particular Vulkan memory
1028	heap.
1029
1030	Any of the elements may be equal to `VK_WHOLE_SIZE`, which means no limit on that
1031	heap. This is also the default in case of `pHeapSizeLimit` = NULL.
1032
1033	If there is a limit defined for a heap:
1034
1035	- If user tries to allocate more memory from that heap using this allocator,
1036	the allocation fails with `VK_ERROR_OUT_OF_DEVICE_MEMORY`.
1037	- If the limit is smaller than heap size reported in `VkMemoryHeap::size`, the
1038	value of this limit will be reported instead when using vmaGetMemoryProperties().
1039
1040	Warning! Using this feature may not be equivalent to installing a GPU with
1041	smaller amount of memory, because graphics driver doesn't necessary fail new
1042	allocations with `VK_ERROR_OUT_OF_DEVICE_MEMORY` result when memory capacity is
1043	exceeded. It may return success and just silently migrate some device memory
1044	blocks to system RAM. This driver behavior can also be controlled using
1045	VK_AMD_memory_overallocation_behavior extension.
1046	*/
1047	const VkDeviceSize* VMA_NULLABLE VMA_LEN_IF_NOT_NULL("VkPhysicalDeviceMemoryProperties::memoryHeapCount") pHeapSizeLimit;
1048
1049	/* \brief Pointers to Vulkan functions. Can be null.*
1050
1051	For details see [Pointers to Vulkan functions](@ref config_Vulkan_functions).
1052	*/
1053	const VmaVulkanFunctions* VMA_NULLABLE pVulkanFunctions;
1054	/* \brief Handle to Vulkan instance object.*
1055
1056	Starting from version 3.0.0 this member is no longer optional, it must be set!
1057	*/
1058	VkInstance VMA_NOT_NULL instance;
1059	/* \brief Optional. The highest version of Vulkan that the application is designed to use.*
1060
1061	It must be a value in the format as created by macro `VK_MAKE_VERSION` or a constant like: `VK_API_VERSION_1_1`, `VK_API_VERSION_1_0`.
1062	The patch version number specified is ignored. Only the major and minor versions are considered.
1063	It must be less or equal (preferably equal) to value as passed to `vkCreateInstance` as `VkApplicationInfo::apiVersion`.
1064	Only versions 1.0, 1.1, 1.2, 1.3 are supported by the current implementation.
1065	Leaving it initialized to zero is equivalent to `VK_API_VERSION_1_0`.
1066	*/
1067	uint32_t vulkanApiVersion;
1068	#if VMA_EXTERNAL_MEMORY
1069	/* \brief Either null or a pointer to an array of external memory handle types for each Vulkan memory type.*
1070
1071	If not NULL, it must be a pointer to an array of `VkPhysicalDeviceMemoryProperties::memoryTypeCount`
1072	elements, defining external memory handle types of particular Vulkan memory type,
1073	to be passed using `VkExportMemoryAllocateInfoKHR`.
1074
1075	Any of the elements may be equal to 0, which means not to use `VkExportMemoryAllocateInfoKHR` on this memory type.
1076	This is also the default in case of `pTypeExternalMemoryHandleTypes` = NULL.
1077	*/
1078	const VkExternalMemoryHandleTypeFlagsKHR* VMA_NULLABLE VMA_LEN_IF_NOT_NULL("VkPhysicalDeviceMemoryProperties::memoryTypeCount") pTypeExternalMemoryHandleTypes;
1079	#endif // #if VMA_EXTERNAL_MEMORY
1080	} VmaAllocatorCreateInfo;
1081
1082	/// Information about existing #VmaAllocator object.
1083	typedef struct VmaAllocatorInfo
1084	{
1085	/* \brief Handle to Vulkan instance object.*
1086
1087	This is the same value as has been passed through VmaAllocatorCreateInfo::instance.
1088	*/
1089	VkInstance VMA_NOT_NULL instance;
1090	/* \brief Handle to Vulkan physical device object.*
1091
1092	This is the same value as has been passed through VmaAllocatorCreateInfo::physicalDevice.
1093	*/
1094	VkPhysicalDevice VMA_NOT_NULL physicalDevice;
1095	/* \brief Handle to Vulkan device object.*
1096
1097	This is the same value as has been passed through VmaAllocatorCreateInfo::device.
1098	*/
1099	VkDevice VMA_NOT_NULL device;
1100	} VmaAllocatorInfo;
1101
1102	/* @} /
1103
1104	/**
1105	\addtogroup group_stats
1106	@{
1107	*/
1108
1109	/* \brief Calculated statistics of memory usage e.g. in a specific memory type, heap, custom pool, or total.*
1110
1111	These are fast to calculate.
1112	See functions: vmaGetHeapBudgets(), vmaGetPoolStatistics().
1113	*/
1114	typedef struct VmaStatistics
1115	{
1116	/* \brief Number of `VkDeviceMemory` objects - Vulkan memory blocks allocated.*
1117	*/
1118	uint32_t blockCount;
1119	/* \brief Number of #VmaAllocation objects allocated.*
1120
1121	Dedicated allocations have their own blocks, so each one adds 1 to `allocationCount` as well as `blockCount`.
1122	*/
1123	uint32_t allocationCount;
1124	/* \brief Number of bytes allocated in `VkDeviceMemory` blocks.*
1125
1126	\note To avoid confusion, please be aware that what Vulkan calls an "allocation" - a whole `VkDeviceMemory` object
1127	(e.g. as in `VkPhysicalDeviceLimits::maxMemoryAllocationCount`) is called a "block" in VMA, while VMA calls
1128	"allocation" a #VmaAllocation object that represents a memory region sub-allocated from such block, usually for a single buffer or image.
1129	*/
1130	VkDeviceSize blockBytes;
1131	/* \brief Total number of bytes occupied by all #VmaAllocation objects.*
1132
1133	Always less or equal than `blockBytes`.
1134	Difference `(blockBytes - allocationBytes)` is the amount of memory allocated from Vulkan
1135	but unused by any #VmaAllocation.
1136	*/
1137	VkDeviceSize allocationBytes;
1138	} VmaStatistics;
1139
1140	/* \brief More detailed statistics than #VmaStatistics.*
1141
1142	These are slower to calculate. Use for debugging purposes.
1143	See functions: vmaCalculateStatistics(), vmaCalculatePoolStatistics().
1144
1145	Previous version of the statistics API provided averages, but they have been removed
1146	because they can be easily calculated as:
1147
1148	\code
1149	VkDeviceSize allocationSizeAvg = detailedStats.statistics.allocationBytes / detailedStats.statistics.allocationCount;
1150	VkDeviceSize unusedBytes = detailedStats.statistics.blockBytes - detailedStats.statistics.allocationBytes;
1151	VkDeviceSize unusedRangeSizeAvg = unusedBytes / detailedStats.unusedRangeCount;
1152	\endcode
1153	*/
1154	typedef struct VmaDetailedStatistics
1155	{
1156	/// Basic statistics.
1157	VmaStatistics statistics;
1158	/// Number of free ranges of memory between allocations.
1159	uint32_t unusedRangeCount;
1160	/// Smallest allocation size. `VK_WHOLE_SIZE` if there are 0 allocations.
1161	VkDeviceSize allocationSizeMin;
1162	/// Largest allocation size. 0 if there are 0 allocations.
1163	VkDeviceSize allocationSizeMax;
1164	/// Smallest empty range size. `VK_WHOLE_SIZE` if there are 0 empty ranges.
1165	VkDeviceSize unusedRangeSizeMin;
1166	/// Largest empty range size. 0 if there are 0 empty ranges.
1167	VkDeviceSize unusedRangeSizeMax;
1168	} VmaDetailedStatistics;
1169
1170	/* \brief General statistics from current state of the Allocator -*
1171	total memory usage across all memory heaps and types.
1172
1173	These are slower to calculate. Use for debugging purposes.
1174	See function vmaCalculateStatistics().
1175	*/
1176	typedef struct VmaTotalStatistics
1177	{
1178	VmaDetailedStatistics memoryType[VK_MAX_MEMORY_TYPES];
1179	VmaDetailedStatistics memoryHeap[VK_MAX_MEMORY_HEAPS];
1180	VmaDetailedStatistics total;
1181	} VmaTotalStatistics;
1182
1183	/* \brief Statistics of current memory usage and available budget for a specific memory heap.*
1184
1185	These are fast to calculate.
1186	See function vmaGetHeapBudgets().
1187	*/
1188	typedef struct VmaBudget
1189	{
1190	/* \brief Statistics fetched from the library.*
1191	*/
1192	VmaStatistics statistics;
1193	/* \brief Estimated current memory usage of the program, in bytes.*
1194
1195	Fetched from system using VK_EXT_memory_budget extension if enabled.
1196
1197	It might be different than `statistics.blockBytes` (usually higher) due to additional implicit objects
1198	also occupying the memory, like swapchain, pipelines, descriptor heaps, command buffers, or
1199	`VkDeviceMemory` blocks allocated outside of this library, if any.
1200	*/
1201	VkDeviceSize usage;
1202	/* \brief Estimated amount of memory available to the program, in bytes.*
1203
1204	Fetched from system using VK_EXT_memory_budget extension if enabled.
1205
1206	It might be different (most probably smaller) than `VkMemoryHeap::size[heapIndex]` due to factors
1207	external to the program, decided by the operating system.
1208	Difference `budget - usage` is the amount of additional memory that can probably
1209	be allocated without problems. Exceeding the budget may result in various problems.
1210	*/
1211	VkDeviceSize budget;
1212	} VmaBudget;
1213
1214	/* @} /
1215
1216	/**
1217	\addtogroup group_alloc
1218	@{
1219	*/
1220
1221	/* \brief Parameters of new #VmaAllocation.*
1222
1223	To be used with functions like vmaCreateBuffer(), vmaCreateImage(), and many others.
1224	*/
1225	typedef struct VmaAllocationCreateInfo
1226	{
1227	/// Use #VmaAllocationCreateFlagBits enum.
1228	VmaAllocationCreateFlags flags;
1229	/* \brief Intended usage of memory.*
1230
1231	You can leave #VMA_MEMORY_USAGE_UNKNOWN if you specify memory requirements in other way. \n
1232	If `pool` is not null, this member is ignored.
1233	*/
1234	VmaMemoryUsage usage;
1235	/* \brief Flags that must be set in a Memory Type chosen for an allocation.*
1236
1237	Leave 0 if you specify memory requirements in other way. \n
1238	If `pool` is not null, this member is ignored./*
1239	VkMemoryPropertyFlags requiredFlags;
1240	/* \brief Flags that preferably should be set in a memory type chosen for an allocation.*
1241
1242	Set to 0 if no additional flags are preferred. \n
1243	If `pool` is not null, this member is ignored. /*
1244	VkMemoryPropertyFlags preferredFlags;
1245	/* \brief Bitmask containing one bit set for every memory type acceptable for this allocation.*
1246
1247	Value 0 is equivalent to `UINT32_MAX` - it means any memory type is accepted if
1248	it meets other requirements specified by this structure, with no further
1249	restrictions on memory type index. \n
1250	If `pool` is not null, this member is ignored.
1251	*/
1252	uint32_t memoryTypeBits;
1253	/* \brief Pool that this allocation should be created in.*
1254
1255	Leave `VK_NULL_HANDLE` to allocate from default pool. If not null, members:
1256	`usage`, `requiredFlags`, `preferredFlags`, `memoryTypeBits` are ignored.
1257	*/
1258	VmaPool VMA_NULLABLE pool;
1259	/* \brief Custom general-purpose pointer that will be stored in #VmaAllocation, can be read as VmaAllocationInfo::pUserData and changed using vmaSetAllocationUserData().*
1260
1261	If #VMA_ALLOCATION_CREATE_USER_DATA_COPY_STRING_BIT is used, it must be either
1262	null or pointer to a null-terminated string. The string will be then copied to
1263	internal buffer, so it doesn't need to be valid after allocation call.
1264	*/
1265	void* VMA_NULLABLE pUserData;
1266	/* \brief A floating-point value between 0 and 1, indicating the priority of the allocation relative to other memory allocations.*
1267
1268	It is used only when #VMA_ALLOCATOR_CREATE_EXT_MEMORY_PRIORITY_BIT flag was used during creation of the #VmaAllocator object
1269	and this allocation ends up as dedicated or is explicitly forced as dedicated using #VMA_ALLOCATION_CREATE_DEDICATED_MEMORY_BIT.
1270	Otherwise, it has the priority of a memory block where it is placed and this variable is ignored.
1271	*/
1272	float priority;
1273	} VmaAllocationCreateInfo;
1274
1275	/// Describes parameter of created #VmaPool.
1276	typedef struct VmaPoolCreateInfo
1277	{
1278	/* \brief Vulkan memory type index to allocate this pool from.*
1279	*/
1280	uint32_t memoryTypeIndex;
1281	/* \brief Use combination of #VmaPoolCreateFlagBits.*
1282	*/
1283	VmaPoolCreateFlags flags;
1284	/* \brief Size of a single `VkDeviceMemory` block to be allocated as part of this pool, in bytes. Optional.*
1285
1286	Specify nonzero to set explicit, constant size of memory blocks used by this
1287	pool.
1288
1289	Leave 0 to use default and let the library manage block sizes automatically.
1290	Sizes of particular blocks may vary.
1291	In this case, the pool will also support dedicated allocations.
1292	*/
1293	VkDeviceSize blockSize;
1294	/* \brief Minimum number of blocks to be always allocated in this pool, even if they stay empty.*
1295
1296	Set to 0 to have no preallocated blocks and allow the pool be completely empty.
1297	*/
1298	size_t minBlockCount;
1299	/* \brief Maximum number of blocks that can be allocated in this pool. Optional.*
1300
1301	Set to 0 to use default, which is `SIZE_MAX`, which means no limit.
1302
1303	Set to same value as VmaPoolCreateInfo::minBlockCount to have fixed amount of memory allocated
1304	throughout whole lifetime of this pool.
1305	*/
1306	size_t maxBlockCount;
1307	/* \brief A floating-point value between 0 and 1, indicating the priority of the allocations in this pool relative to other memory allocations.*
1308
1309	It is used only when #VMA_ALLOCATOR_CREATE_EXT_MEMORY_PRIORITY_BIT flag was used during creation of the #VmaAllocator object.
1310	Otherwise, this variable is ignored.
1311	*/
1312	float priority;
1313	/* \brief Additional minimum alignment to be used for all allocations created from this pool. Can be 0.*
1314
1315	Leave 0 (default) not to impose any additional alignment. If not 0, it must be a power of two.
1316	It can be useful in cases where alignment returned by Vulkan by functions like `vkGetBufferMemoryRequirements` is not enough,
1317	e.g. when doing interop with OpenGL.
1318	*/
1319	VkDeviceSize minAllocationAlignment;
1320	/* \brief Additional `pNext` chain to be attached to `VkMemoryAllocateInfo` used for every allocation made by this pool. Optional.*
1321
1322	Optional, can be null. If not null, it must point to a `pNext` chain of structures that can be attached to `VkMemoryAllocateInfo`.
1323	It can be useful for special needs such as adding `VkExportMemoryAllocateInfoKHR`.
1324	Structures pointed by this member must remain alive and unchanged for the whole lifetime of the custom pool.
1325
1326	Please note that some structures, e.g. `VkMemoryPriorityAllocateInfoEXT`, `VkMemoryDedicatedAllocateInfoKHR`,
1327	can be attached automatically by this library when using other, more convenient of its features.
1328	*/
1329	void* VMA_NULLABLE pMemoryAllocateNext;
1330	} VmaPoolCreateInfo;
1331
1332	/* @} /
1333
1334	/**
1335	\addtogroup group_alloc
1336	@{
1337	*/
1338
1339	/// Parameters of #VmaAllocation objects, that can be retrieved using function vmaGetAllocationInfo().
1340	typedef struct VmaAllocationInfo
1341	{
1342	/* \brief Memory type index that this allocation was allocated from.*
1343
1344	It never changes.
1345	*/
1346	uint32_t memoryType;
1347	/* \brief Handle to Vulkan memory object.*
1348
1349	Same memory object can be shared by multiple allocations.
1350
1351	It can change after the allocation is moved during \ref defragmentation.
1352	*/
1353	VkDeviceMemory VMA_NULLABLE_NON_DISPATCHABLE deviceMemory;
1354	/* \brief Offset in `VkDeviceMemory` object to the beginning of this allocation, in bytes. `(deviceMemory, offset)` pair is unique to this allocation.*
1355
1356	You usually don't need to use this offset. If you create a buffer or an image together with the allocation using e.g. function
1357	vmaCreateBuffer(), vmaCreateImage(), functions that operate on these resources refer to the beginning of the buffer or image,
1358	not entire device memory block. Functions like vmaMapMemory(), vmaBindBufferMemory() also refer to the beginning of the allocation
1359	and apply this offset automatically.
1360
1361	It can change after the allocation is moved during \ref defragmentation.
1362	*/
1363	VkDeviceSize offset;
1364	/* \brief Size of this allocation, in bytes.*
1365
1366	It never changes.
1367
1368	\note Allocation size returned in this variable may be greater than the size
1369	requested for the resource e.g. as `VkBufferCreateInfo::size`. Whole size of the
1370	allocation is accessible for operations on memory e.g. using a pointer after
1371	mapping with vmaMapMemory(), but operations on the resource e.g. using
1372	`vkCmdCopyBuffer` must be limited to the size of the resource.
1373	*/
1374	VkDeviceSize size;
1375	/* \brief Pointer to the beginning of this allocation as mapped data.*
1376
1377	If the allocation hasn't been mapped using vmaMapMemory() and hasn't been
1378	created with #VMA_ALLOCATION_CREATE_MAPPED_BIT flag, this value is null.
1379
1380	It can change after call to vmaMapMemory(), vmaUnmapMemory().
1381	It can also change after the allocation is moved during \ref defragmentation.
1382	*/
1383	void* VMA_NULLABLE pMappedData;
1384	/* \brief Custom general-purpose pointer that was passed as VmaAllocationCreateInfo::pUserData or set using vmaSetAllocationUserData().*
1385
1386	It can change after call to vmaSetAllocationUserData() for this allocation.
1387	*/
1388	void* VMA_NULLABLE pUserData;
1389	/* \brief Custom allocation name that was set with vmaSetAllocationName().*
1390
1391	It can change after call to vmaSetAllocationName() for this allocation.
1392
1393	Another way to set custom name is to pass it in VmaAllocationCreateInfo::pUserData with
1394	additional flag #VMA_ALLOCATION_CREATE_USER_DATA_COPY_STRING_BIT set [DEPRECATED].
1395	*/
1396	const char* VMA_NULLABLE pName;
1397	} VmaAllocationInfo;
1398
1399	/* \brief Parameters for defragmentation.*
1400
1401	To be used with function vmaBeginDefragmentation().
1402	*/
1403	typedef struct VmaDefragmentationInfo
1404	{
1405	/// \brief Use combination of #VmaDefragmentationFlagBits.
1406	VmaDefragmentationFlags flags;
1407	/* \brief Custom pool to be defragmented.*
1408
1409	If null then default pools will undergo defragmentation process.
1410	*/
1411	VmaPool VMA_NULLABLE pool;
1412	/* \brief Maximum numbers of bytes that can be copied during single pass, while moving allocations to different places.*
1413
1414	`0` means no limit.
1415	*/
1416	VkDeviceSize maxBytesPerPass;
1417	/* \brief Maximum number of allocations that can be moved during single pass to a different place.*
1418
1419	`0` means no limit.
1420	*/
1421	uint32_t maxAllocationsPerPass;
1422	} VmaDefragmentationInfo;
1423
1424	/// Single move of an allocation to be done for defragmentation.
1425	typedef struct VmaDefragmentationMove
1426	{
1427	/// Operation to be performed on the allocation by vmaEndDefragmentationPass(). Default value is #VMA_DEFRAGMENTATION_MOVE_OPERATION_COPY. You can modify it.
1428	VmaDefragmentationMoveOperation operation;
1429	/// Allocation that should be moved.
1430	VmaAllocation VMA_NOT_NULL srcAllocation;
1431	/* \brief Temporary allocation pointing to destination memory that will replace `srcAllocation`.*
1432
1433	\warning Do not store this allocation in your data structures! It exists only temporarily, for the duration of the defragmentation pass,
1434	to be used for binding new buffer/image to the destination memory using e.g. vmaBindBufferMemory().
1435	vmaEndDefragmentationPass() will destroy it and make `srcAllocation` point to this memory.
1436	*/
1437	VmaAllocation VMA_NOT_NULL dstTmpAllocation;
1438	} VmaDefragmentationMove;
1439
1440	/* \brief Parameters for incremental defragmentation steps.*
1441
1442	To be used with function vmaBeginDefragmentationPass().
1443	*/
1444	typedef struct VmaDefragmentationPassMoveInfo
1445	{
1446	/// Number of elements in the `pMoves` array.
1447	uint32_t moveCount;
1448	/* \brief Array of moves to be performed by the user in the current defragmentation pass.*
1449
1450	Pointer to an array of `moveCount` elements, owned by VMA, created in vmaBeginDefragmentationPass(), destroyed in vmaEndDefragmentationPass().
1451
1452	For each element, you should:
1453
1454	1. Create a new buffer/image in the place pointed by VmaDefragmentationMove::dstMemory + VmaDefragmentationMove::dstOffset.
1455	2. Copy data from the VmaDefragmentationMove::srcAllocation e.g. using `vkCmdCopyBuffer`, `vkCmdCopyImage`.
1456	3. Make sure these commands finished executing on the GPU.
1457	4. Destroy the old buffer/image.
1458
1459	Only then you can finish defragmentation pass by calling vmaEndDefragmentationPass().
1460	After this call, the allocation will point to the new place in memory.
1461
1462	Alternatively, if you cannot move specific allocation, you can set VmaDefragmentationMove::operation to #VMA_DEFRAGMENTATION_MOVE_OPERATION_IGNORE.
1463
1464	Alternatively, if you decide you want to completely remove the allocation:
1465
1466	1. Destroy its buffer/image.
1467	2. Set VmaDefragmentationMove::operation to #VMA_DEFRAGMENTATION_MOVE_OPERATION_DESTROY.
1468
1469	Then, after vmaEndDefragmentationPass() the allocation will be freed.
1470	*/
1471	VmaDefragmentationMove* VMA_NULLABLE VMA_LEN_IF_NOT_NULL(moveCount) pMoves;
1472	} VmaDefragmentationPassMoveInfo;
1473
1474	/// Statistics returned for defragmentation process in function vmaEndDefragmentation().
1475	typedef struct VmaDefragmentationStats
1476	{
1477	/// Total number of bytes that have been copied while moving allocations to different places.
1478	VkDeviceSize bytesMoved;
1479	/// Total number of bytes that have been released to the system by freeing empty `VkDeviceMemory` objects.
1480	VkDeviceSize bytesFreed;
1481	/// Number of allocations that have been moved to different places.
1482	uint32_t allocationsMoved;
1483	/// Number of empty `VkDeviceMemory` objects that have been released to the system.
1484	uint32_t deviceMemoryBlocksFreed;
1485	} VmaDefragmentationStats;
1486
1487	/* @} /
1488
1489	/**
1490	\addtogroup group_virtual
1491	@{
1492	*/
1493
1494	/// Parameters of created #VmaVirtualBlock object to be passed to vmaCreateVirtualBlock().
1495	typedef struct VmaVirtualBlockCreateInfo
1496	{
1497	/* \brief Total size of the virtual block.*
1498
1499	Sizes can be expressed in bytes or any units you want as long as you are consistent in using them.
1500	For example, if you allocate from some array of structures, 1 can mean single instance of entire structure.
1501	*/
1502	VkDeviceSize size;
1503
1504	/* \brief Use combination of #VmaVirtualBlockCreateFlagBits.*
1505	*/
1506	VmaVirtualBlockCreateFlags flags;
1507
1508	/* \brief Custom CPU memory allocation callbacks. Optional.*
1509
1510	Optional, can be null. When specified, they will be used for all CPU-side memory allocations.
1511	*/
1512	const VkAllocationCallbacks* VMA_NULLABLE pAllocationCallbacks;
1513	} VmaVirtualBlockCreateInfo;
1514
1515	/// Parameters of created virtual allocation to be passed to vmaVirtualAllocate().
1516	typedef struct VmaVirtualAllocationCreateInfo
1517	{
1518	/* \brief Size of the allocation.*
1519
1520	Cannot be zero.
1521	*/
1522	VkDeviceSize size;
1523	/* \brief Required alignment of the allocation. Optional.*
1524
1525	Must be power of two. Special value 0 has the same meaning as 1 - means no special alignment is required, so allocation can start at any offset.
1526	*/
1527	VkDeviceSize alignment;
1528	/* \brief Use combination of #VmaVirtualAllocationCreateFlagBits.*
1529	*/
1530	VmaVirtualAllocationCreateFlags flags;
1531	/* \brief Custom pointer to be associated with the allocation. Optional.*
1532
1533	It can be any value and can be used for user-defined purposes. It can be fetched or changed later.
1534	*/
1535	void* VMA_NULLABLE pUserData;
1536	} VmaVirtualAllocationCreateInfo;
1537
1538	/// Parameters of an existing virtual allocation, returned by vmaGetVirtualAllocationInfo().
1539	typedef struct VmaVirtualAllocationInfo
1540	{
1541	/* \brief Offset of the allocation.*
1542
1543	Offset at which the allocation was made.
1544	*/
1545	VkDeviceSize offset;
1546	/* \brief Size of the allocation.*
1547
1548	Same value as passed in VmaVirtualAllocationCreateInfo::size.
1549	*/
1550	VkDeviceSize size;
1551	/* \brief Custom pointer associated with the allocation.*
1552
1553	Same value as passed in VmaVirtualAllocationCreateInfo::pUserData or to vmaSetVirtualAllocationUserData().
1554	*/
1555	void* VMA_NULLABLE pUserData;
1556	} VmaVirtualAllocationInfo;
1557
1558	/* @} /
1559
1560	#endif // _VMA_DATA_TYPES_DECLARATIONS
1561
1562	#ifndef _VMA_FUNCTION_HEADERS
1563
1564	/**
1565	\addtogroup group_init
1566	@{
1567	*/
1568
1569	/// Creates #VmaAllocator object.
1570	VMA_CALL_PRE VkResult VMA_CALL_POST vmaCreateAllocator(
1571	const VmaAllocatorCreateInfo* VMA_NOT_NULL pCreateInfo,
1572	VmaAllocator VMA_NULLABLE* VMA_NOT_NULL pAllocator);
1573
1574	/// Destroys allocator object.
1575	VMA_CALL_PRE void VMA_CALL_POST vmaDestroyAllocator(
1576	VmaAllocator VMA_NULLABLE allocator);
1577
1578	/* \brief Returns information about existing #VmaAllocator object - handle to Vulkan device etc.*
1579
1580	It might be useful if you want to keep just the #VmaAllocator handle and fetch other required handles to
1581	`VkPhysicalDevice`, `VkDevice` etc. every time using this function.
1582	*/
1583	VMA_CALL_PRE void VMA_CALL_POST vmaGetAllocatorInfo(
1584	VmaAllocator VMA_NOT_NULL allocator,
1585	VmaAllocatorInfo* VMA_NOT_NULL pAllocatorInfo);
1586
1587	/**
1588	PhysicalDeviceProperties are fetched from physicalDevice by the allocator.
1589	You can access it here, without fetching it again on your own.
1590	*/
1591	VMA_CALL_PRE void VMA_CALL_POST vmaGetPhysicalDeviceProperties(
1592	VmaAllocator VMA_NOT_NULL allocator,
1593	const VkPhysicalDeviceProperties* VMA_NULLABLE* VMA_NOT_NULL ppPhysicalDeviceProperties);
1594
1595	/**
1596	PhysicalDeviceMemoryProperties are fetched from physicalDevice by the allocator.
1597	You can access it here, without fetching it again on your own.
1598	*/
1599	VMA_CALL_PRE void VMA_CALL_POST vmaGetMemoryProperties(
1600	VmaAllocator VMA_NOT_NULL allocator,
1601	const VkPhysicalDeviceMemoryProperties* VMA_NULLABLE* VMA_NOT_NULL ppPhysicalDeviceMemoryProperties);
1602
1603	/**
1604	\brief Given Memory Type Index, returns Property Flags of this memory type.
1605
1606	This is just a convenience function. Same information can be obtained using
1607	vmaGetMemoryProperties().
1608	*/
1609	VMA_CALL_PRE void VMA_CALL_POST vmaGetMemoryTypeProperties(
1610	VmaAllocator VMA_NOT_NULL allocator,
1611	uint32_t memoryTypeIndex,
1612	VkMemoryPropertyFlags* VMA_NOT_NULL pFlags);
1613
1614	/* \brief Sets index of the current frame.*
1615	*/
1616	VMA_CALL_PRE void VMA_CALL_POST vmaSetCurrentFrameIndex(
1617	VmaAllocator VMA_NOT_NULL allocator,
1618	uint32_t frameIndex);
1619
1620	/* @} /
1621
1622	/**
1623	\addtogroup group_stats
1624	@{
1625	*/
1626
1627	/* \brief Retrieves statistics from current state of the Allocator.*
1628
1629	This function is called "calculate" not "get" because it has to traverse all
1630	internal data structures, so it may be quite slow. Use it for debugging purposes.
1631	For faster but more brief statistics suitable to be called every frame or every allocation,
1632	use vmaGetHeapBudgets().
1633
1634	Note that when using allocator from multiple threads, returned information may immediately
1635	become outdated.
1636	*/
1637	VMA_CALL_PRE void VMA_CALL_POST vmaCalculateStatistics(
1638	VmaAllocator VMA_NOT_NULL allocator,
1639	VmaTotalStatistics* VMA_NOT_NULL pStats);
1640
1641	/* \brief Retrieves information about current memory usage and budget for all memory heaps.*
1642
1643	\param allocator
1644	\param[out] pBudgets Must point to array with number of elements at least equal to number of memory heaps in physical device used.
1645
1646	This function is called "get" not "calculate" because it is very fast, suitable to be called
1647	every frame or every allocation. For more detailed statistics use vmaCalculateStatistics().
1648
1649	Note that when using allocator from multiple threads, returned information may immediately
1650	become outdated.
1651	*/
1652	VMA_CALL_PRE void VMA_CALL_POST vmaGetHeapBudgets(
1653	VmaAllocator VMA_NOT_NULL allocator,
1654	VmaBudget* VMA_NOT_NULL VMA_LEN_IF_NOT_NULL("VkPhysicalDeviceMemoryProperties::memoryHeapCount") pBudgets);
1655
1656	/* @} /
1657
1658	/**
1659	\addtogroup group_alloc
1660	@{
1661	*/
1662
1663	/**
1664	\brief Helps to find memoryTypeIndex, given memoryTypeBits and VmaAllocationCreateInfo.
1665
1666	This algorithm tries to find a memory type that:
1667
1668	- Is allowed by memoryTypeBits.
1669	- Contains all the flags from pAllocationCreateInfo->requiredFlags.
1670	- Matches intended usage.
1671	- Has as many flags from pAllocationCreateInfo->preferredFlags as possible.
1672
1673	\return Returns VK_ERROR_FEATURE_NOT_PRESENT if not found. Receiving such result
1674	from this function or any other allocating function probably means that your
1675	device doesn't support any memory type with requested features for the specific
1676	type of resource you want to use it for. Please check parameters of your
1677	resource, like image layout (OPTIMAL versus LINEAR) or mip level count.
1678	*/
1679	VMA_CALL_PRE VkResult VMA_CALL_POST vmaFindMemoryTypeIndex(
1680	VmaAllocator VMA_NOT_NULL allocator,
1681	uint32_t memoryTypeBits,
1682	const VmaAllocationCreateInfo* VMA_NOT_NULL pAllocationCreateInfo,
1683	uint32_t* VMA_NOT_NULL pMemoryTypeIndex);
1684
1685	/**
1686	\brief Helps to find memoryTypeIndex, given VkBufferCreateInfo and VmaAllocationCreateInfo.
1687
1688	It can be useful e.g. to determine value to be used as VmaPoolCreateInfo::memoryTypeIndex.
1689	It internally creates a temporary, dummy buffer that never has memory bound.
1690	*/
1691	VMA_CALL_PRE VkResult VMA_CALL_POST vmaFindMemoryTypeIndexForBufferInfo(
1692	VmaAllocator VMA_NOT_NULL allocator,
1693	const VkBufferCreateInfo* VMA_NOT_NULL pBufferCreateInfo,
1694	const VmaAllocationCreateInfo* VMA_NOT_NULL pAllocationCreateInfo,
1695	uint32_t* VMA_NOT_NULL pMemoryTypeIndex);
1696
1697	/**
1698	\brief Helps to find memoryTypeIndex, given VkImageCreateInfo and VmaAllocationCreateInfo.
1699
1700	It can be useful e.g. to determine value to be used as VmaPoolCreateInfo::memoryTypeIndex.
1701	It internally creates a temporary, dummy image that never has memory bound.
1702	*/
1703	VMA_CALL_PRE VkResult VMA_CALL_POST vmaFindMemoryTypeIndexForImageInfo(
1704	VmaAllocator VMA_NOT_NULL allocator,
1705	const VkImageCreateInfo* VMA_NOT_NULL pImageCreateInfo,
1706	const VmaAllocationCreateInfo* VMA_NOT_NULL pAllocationCreateInfo,
1707	uint32_t* VMA_NOT_NULL pMemoryTypeIndex);
1708
1709	/* \brief Allocates Vulkan device memory and creates #VmaPool object.*
1710
1711	\param allocator Allocator object.
1712	\param pCreateInfo Parameters of pool to create.
1713	\param[out] pPool Handle to created pool.
1714	*/
1715	VMA_CALL_PRE VkResult VMA_CALL_POST vmaCreatePool(
1716	VmaAllocator VMA_NOT_NULL allocator,
1717	const VmaPoolCreateInfo* VMA_NOT_NULL pCreateInfo,
1718	VmaPool VMA_NULLABLE* VMA_NOT_NULL pPool);
1719
1720	/* \brief Destroys #VmaPool object and frees Vulkan device memory.*
1721	*/
1722	VMA_CALL_PRE void VMA_CALL_POST vmaDestroyPool(
1723	VmaAllocator VMA_NOT_NULL allocator,
1724	VmaPool VMA_NULLABLE pool);
1725
1726	/* @} /
1727
1728	/**
1729	\addtogroup group_stats
1730	@{
1731	*/
1732
1733	/* \brief Retrieves statistics of existing #VmaPool object.*
1734
1735	\param allocator Allocator object.
1736	\param pool Pool object.
1737	\param[out] pPoolStats Statistics of specified pool.
1738	*/
1739	VMA_CALL_PRE void VMA_CALL_POST vmaGetPoolStatistics(
1740	VmaAllocator VMA_NOT_NULL allocator,
1741	VmaPool VMA_NOT_NULL pool,
1742	VmaStatistics* VMA_NOT_NULL pPoolStats);
1743
1744	/* \brief Retrieves detailed statistics of existing #VmaPool object.*
1745
1746	\param allocator Allocator object.
1747	\param pool Pool object.
1748	\param[out] pPoolStats Statistics of specified pool.
1749	*/
1750	VMA_CALL_PRE void VMA_CALL_POST vmaCalculatePoolStatistics(
1751	VmaAllocator VMA_NOT_NULL allocator,
1752	VmaPool VMA_NOT_NULL pool,
1753	VmaDetailedStatistics* VMA_NOT_NULL pPoolStats);
1754
1755	/* @} /
1756
1757	/**
1758	\addtogroup group_alloc
1759	@{
1760	*/
1761
1762	/* \brief Checks magic number in margins around all allocations in given memory pool in search for corruptions.*
1763
1764	Corruption detection is enabled only when `VMA_DEBUG_DETECT_CORRUPTION` macro is defined to nonzero,
1765	`VMA_DEBUG_MARGIN` is defined to nonzero and the pool is created in memory type that is
1766	`HOST_VISIBLE` and `HOST_COHERENT`. For more information, see [Corruption detection](@ref debugging_memory_usage_corruption_detection).
1767
1768	Possible return values:
1769
1770	- `VK_ERROR_FEATURE_NOT_PRESENT` - corruption detection is not enabled for specified pool.
1771	- `VK_SUCCESS` - corruption detection has been performed and succeeded.
1772	- `VK_ERROR_UNKNOWN` - corruption detection has been performed and found memory corruptions around one of the allocations.
1773	`VMA_ASSERT` is also fired in that case.
1774	- Other value: Error returned by Vulkan, e.g. memory mapping failure.
1775	*/
1776	VMA_CALL_PRE VkResult VMA_CALL_POST vmaCheckPoolCorruption(
1777	VmaAllocator VMA_NOT_NULL allocator,
1778	VmaPool VMA_NOT_NULL pool);
1779
1780	/* \brief Retrieves name of a custom pool.*
1781
1782	After the call `ppName` is either null or points to an internally-owned null-terminated string
1783	containing name of the pool that was previously set. The pointer becomes invalid when the pool is
1784	destroyed or its name is changed using vmaSetPoolName().
1785	*/
1786	VMA_CALL_PRE void VMA_CALL_POST vmaGetPoolName(
1787	VmaAllocator VMA_NOT_NULL allocator,
1788	VmaPool VMA_NOT_NULL pool,
1789	const char* VMA_NULLABLE* VMA_NOT_NULL ppName);
1790
1791	/* \brief Sets name of a custom pool.*
1792
1793	`pName` can be either null or pointer to a null-terminated string with new name for the pool.
1794	Function makes internal copy of the string, so it can be changed or freed immediately after this call.
1795	*/
1796	VMA_CALL_PRE void VMA_CALL_POST vmaSetPoolName(
1797	VmaAllocator VMA_NOT_NULL allocator,
1798	VmaPool VMA_NOT_NULL pool,
1799	const char* VMA_NULLABLE pName);
1800
1801	/* \brief General purpose memory allocation.*
1802
1803	\param allocator
1804	\param pVkMemoryRequirements
1805	\param pCreateInfo
1806	\param[out] pAllocation Handle to allocated memory.
1807	\param[out] pAllocationInfo Optional. Information about allocated memory. It can be later fetched using function vmaGetAllocationInfo().
1808
1809	You should free the memory using vmaFreeMemory() or vmaFreeMemoryPages().
1810
1811	It is recommended to use vmaAllocateMemoryForBuffer(), vmaAllocateMemoryForImage(),
1812	vmaCreateBuffer(), vmaCreateImage() instead whenever possible.
1813	*/
1814	VMA_CALL_PRE VkResult VMA_CALL_POST vmaAllocateMemory(
1815	VmaAllocator VMA_NOT_NULL allocator,
1816	const VkMemoryRequirements* VMA_NOT_NULL pVkMemoryRequirements,
1817	const VmaAllocationCreateInfo* VMA_NOT_NULL pCreateInfo,
1818	VmaAllocation VMA_NULLABLE* VMA_NOT_NULL pAllocation,
1819	VmaAllocationInfo* VMA_NULLABLE pAllocationInfo);
1820
1821	/* \brief General purpose memory allocation for multiple allocation objects at once.*
1822
1823	\param allocator Allocator object.
1824	\param pVkMemoryRequirements Memory requirements for each allocation.
1825	\param pCreateInfo Creation parameters for each allocation.
1826	\param allocationCount Number of allocations to make.
1827	\param[out] pAllocations Pointer to array that will be filled with handles to created allocations.
1828	\param[out] pAllocationInfo Optional. Pointer to array that will be filled with parameters of created allocations.
1829
1830	You should free the memory using vmaFreeMemory() or vmaFreeMemoryPages().
1831
1832	Word "pages" is just a suggestion to use this function to allocate pieces of memory needed for sparse binding.
1833	It is just a general purpose allocation function able to make multiple allocations at once.
1834	It may be internally optimized to be more efficient than calling vmaAllocateMemory() `allocationCount` times.
1835
1836	All allocations are made using same parameters. All of them are created out of the same memory pool and type.
1837	If any allocation fails, all allocations already made within this function call are also freed, so that when
1838	returned result is not `VK_SUCCESS`, `pAllocation` array is always entirely filled with `VK_NULL_HANDLE`.
1839	*/
1840	VMA_CALL_PRE VkResult VMA_CALL_POST vmaAllocateMemoryPages(
1841	VmaAllocator VMA_NOT_NULL allocator,
1842	const VkMemoryRequirements* VMA_NOT_NULL VMA_LEN_IF_NOT_NULL(allocationCount) pVkMemoryRequirements,
1843	const VmaAllocationCreateInfo* VMA_NOT_NULL VMA_LEN_IF_NOT_NULL(allocationCount) pCreateInfo,
1844	size_t allocationCount,
1845	VmaAllocation VMA_NULLABLE* VMA_NOT_NULL VMA_LEN_IF_NOT_NULL(allocationCount) pAllocations,
1846	VmaAllocationInfo* VMA_NULLABLE VMA_LEN_IF_NOT_NULL(allocationCount) pAllocationInfo);
1847
1848	/* \brief Allocates memory suitable for given `VkBuffer`.*
1849
1850	\param allocator
1851	\param buffer
1852	\param pCreateInfo
1853	\param[out] pAllocation Handle to allocated memory.
1854	\param[out] pAllocationInfo Optional. Information about allocated memory. It can be later fetched using function vmaGetAllocationInfo().
1855
1856	It only creates #VmaAllocation. To bind the memory to the buffer, use vmaBindBufferMemory().
1857
1858	This is a special-purpose function. In most cases you should use vmaCreateBuffer().
1859
1860	You must free the allocation using vmaFreeMemory() when no longer needed.
1861	*/
1862	VMA_CALL_PRE VkResult VMA_CALL_POST vmaAllocateMemoryForBuffer(
1863	VmaAllocator VMA_NOT_NULL allocator,
1864	VkBuffer VMA_NOT_NULL_NON_DISPATCHABLE buffer,
1865	const VmaAllocationCreateInfo* VMA_NOT_NULL pCreateInfo,
1866	VmaAllocation VMA_NULLABLE* VMA_NOT_NULL pAllocation,
1867	VmaAllocationInfo* VMA_NULLABLE pAllocationInfo);
1868
1869	/* \brief Allocates memory suitable for given `VkImage`.*
1870
1871	\param allocator
1872	\param image
1873	\param pCreateInfo
1874	\param[out] pAllocation Handle to allocated memory.
1875	\param[out] pAllocationInfo Optional. Information about allocated memory. It can be later fetched using function vmaGetAllocationInfo().
1876
1877	It only creates #VmaAllocation. To bind the memory to the buffer, use vmaBindImageMemory().
1878
1879	This is a special-purpose function. In most cases you should use vmaCreateImage().
1880
1881	You must free the allocation using vmaFreeMemory() when no longer needed.
1882	*/
1883	VMA_CALL_PRE VkResult VMA_CALL_POST vmaAllocateMemoryForImage(
1884	VmaAllocator VMA_NOT_NULL allocator,
1885	VkImage VMA_NOT_NULL_NON_DISPATCHABLE image,
1886	const VmaAllocationCreateInfo* VMA_NOT_NULL pCreateInfo,
1887	VmaAllocation VMA_NULLABLE* VMA_NOT_NULL pAllocation,
1888	VmaAllocationInfo* VMA_NULLABLE pAllocationInfo);
1889
1890	/* \brief Frees memory previously allocated using vmaAllocateMemory(), vmaAllocateMemoryForBuffer(), or vmaAllocateMemoryForImage().*
1891
1892	Passing `VK_NULL_HANDLE` as `allocation` is valid. Such function call is just skipped.
1893	*/
1894	VMA_CALL_PRE void VMA_CALL_POST vmaFreeMemory(
1895	VmaAllocator VMA_NOT_NULL allocator,
1896	const VmaAllocation VMA_NULLABLE allocation);
1897
1898	/* \brief Frees memory and destroys multiple allocations.*
1899
1900	Word "pages" is just a suggestion to use this function to free pieces of memory used for sparse binding.
1901	It is just a general purpose function to free memory and destroy allocations made using e.g. vmaAllocateMemory(),
1902	vmaAllocateMemoryPages() and other functions.
1903	It may be internally optimized to be more efficient than calling vmaFreeMemory() `allocationCount` times.
1904
1905	Allocations in `pAllocations` array can come from any memory pools and types.
1906	Passing `VK_NULL_HANDLE` as elements of `pAllocations` array is valid. Such entries are just skipped.
1907	*/
1908	VMA_CALL_PRE void VMA_CALL_POST vmaFreeMemoryPages(
1909	VmaAllocator VMA_NOT_NULL allocator,
1910	size_t allocationCount,
1911	const VmaAllocation VMA_NULLABLE* VMA_NOT_NULL VMA_LEN_IF_NOT_NULL(allocationCount) pAllocations);
1912
1913	/* \brief Returns current information about specified allocation.*
1914
1915	Current paramteres of given allocation are returned in `pAllocationInfo`.
1916
1917	Although this function doesn't lock any mutex, so it should be quite efficient,
1918	you should avoid calling it too often.
1919	You can retrieve same VmaAllocationInfo structure while creating your resource, from function
1920	vmaCreateBuffer(), vmaCreateImage(). You can remember it if you are sure parameters don't change
1921	(e.g. due to defragmentation).
1922	*/
1923	VMA_CALL_PRE void VMA_CALL_POST vmaGetAllocationInfo(
1924	VmaAllocator VMA_NOT_NULL allocator,
1925	VmaAllocation VMA_NOT_NULL allocation,
1926	VmaAllocationInfo* VMA_NOT_NULL pAllocationInfo);
1927
1928	/* \brief Sets pUserData in given allocation to new value.*
1929
1930	The value of pointer `pUserData` is copied to allocation's `pUserData`.
1931	It is opaque, so you can use it however you want - e.g.
1932	as a pointer, ordinal number or some handle to you own data.
1933	*/
1934	VMA_CALL_PRE void VMA_CALL_POST vmaSetAllocationUserData(
1935	VmaAllocator VMA_NOT_NULL allocator,
1936	VmaAllocation VMA_NOT_NULL allocation,
1937	void* VMA_NULLABLE pUserData);
1938
1939	/* \brief Sets pName in given allocation to new value.*
1940
1941	`pName` must be either null, or pointer to a null-terminated string. The function
1942	makes local copy of the string and sets it as allocation's `pName`. String
1943	passed as pName doesn't need to be valid for whole lifetime of the allocation -
1944	you can free it after this call. String previously pointed by allocation's
1945	`pName` is freed from memory.
1946	*/
1947	VMA_CALL_PRE void VMA_CALL_POST vmaSetAllocationName(
1948	VmaAllocator VMA_NOT_NULL allocator,
1949	VmaAllocation VMA_NOT_NULL allocation,
1950	const char* VMA_NULLABLE pName);
1951
1952	/**
1953	\brief Given an allocation, returns Property Flags of its memory type.
1954
1955	This is just a convenience function. Same information can be obtained using
1956	vmaGetAllocationInfo() + vmaGetMemoryProperties().
1957	*/
1958	VMA_CALL_PRE void VMA_CALL_POST vmaGetAllocationMemoryProperties(
1959	VmaAllocator VMA_NOT_NULL allocator,
1960	VmaAllocation VMA_NOT_NULL allocation,
1961	VkMemoryPropertyFlags* VMA_NOT_NULL pFlags);
1962
1963	/* \brief Maps memory represented by given allocation and returns pointer to it.*
1964
1965	Maps memory represented by given allocation to make it accessible to CPU code.
1966	When succeeded, `ppData` contains pointer to first byte of this memory.*
1967
1968	\warning
1969	If the allocation is part of a bigger `VkDeviceMemory` block, returned pointer is
1970	correctly offsetted to the beginning of region assigned to this particular allocation.
1971	Unlike the result of `vkMapMemory`, it points to the allocation, not to the beginning of the whole block.
1972	You should not add VmaAllocationInfo::offset to it!
1973
1974	Mapping is internally reference-counted and synchronized, so despite raw Vulkan
1975	function `vkMapMemory()` cannot be used to map same block of `VkDeviceMemory`
1976	multiple times simultaneously, it is safe to call this function on allocations
1977	assigned to the same memory block. Actual Vulkan memory will be mapped on first
1978	mapping and unmapped on last unmapping.
1979
1980	If the function succeeded, you must call vmaUnmapMemory() to unmap the
1981	allocation when mapping is no longer needed or before freeing the allocation, at
1982	the latest.
1983
1984	It also safe to call this function multiple times on the same allocation. You
1985	must call vmaUnmapMemory() same number of times as you called vmaMapMemory().
1986
1987	It is also safe to call this function on allocation created with
1988	#VMA_ALLOCATION_CREATE_MAPPED_BIT flag. Its memory stays mapped all the time.
1989	You must still call vmaUnmapMemory() same number of times as you called
1990	vmaMapMemory(). You must not call vmaUnmapMemory() additional time to free the
1991	"0-th" mapping made automatically due to #VMA_ALLOCATION_CREATE_MAPPED_BIT flag.
1992
1993	This function fails when used on allocation made in memory type that is not
1994	`HOST_VISIBLE`.
1995
1996	This function doesn't automatically flush or invalidate caches.
1997	If the allocation is made from a memory types that is not `HOST_COHERENT`,
1998	you also need to use vmaInvalidateAllocation() / vmaFlushAllocation(), as required by Vulkan specification.
1999	*/
2000	VMA_CALL_PRE VkResult VMA_CALL_POST vmaMapMemory(
2001	VmaAllocator VMA_NOT_NULL allocator,
2002	VmaAllocation VMA_NOT_NULL allocation,
2003	void* VMA_NULLABLE* VMA_NOT_NULL ppData);
2004
2005	/* \brief Unmaps memory represented by given allocation, mapped previously using vmaMapMemory().*
2006
2007	For details, see description of vmaMapMemory().
2008
2009	This function doesn't automatically flush or invalidate caches.
2010	If the allocation is made from a memory types that is not `HOST_COHERENT`,
2011	you also need to use vmaInvalidateAllocation() / vmaFlushAllocation(), as required by Vulkan specification.
2012	*/
2013	VMA_CALL_PRE void VMA_CALL_POST vmaUnmapMemory(
2014	VmaAllocator VMA_NOT_NULL allocator,
2015	VmaAllocation VMA_NOT_NULL allocation);
2016
2017	/* \brief Flushes memory of given allocation.*
2018
2019	Calls `vkFlushMappedMemoryRanges()` for memory associated with given range of given allocation.
2020	It needs to be called after writing to a mapped memory for memory types that are not `HOST_COHERENT`.
2021	Unmap operation doesn't do that automatically.
2022
2023	- `offset` must be relative to the beginning of allocation.
2024	- `size` can be `VK_WHOLE_SIZE`. It means all memory from `offset` the the end of given allocation.
2025	- `offset` and `size` don't have to be aligned.
2026	They are internally rounded down/up to multiply of `nonCoherentAtomSize`.
2027	- If `size` is 0, this call is ignored.
2028	- If memory type that the `allocation` belongs to is not `HOST_VISIBLE` or it is `HOST_COHERENT`,
2029	this call is ignored.
2030
2031	Warning! `offset` and `size` are relative to the contents of given `allocation`.
2032	If you mean whole allocation, you can pass 0 and `VK_WHOLE_SIZE`, respectively.
2033	Do not pass allocation's offset as `offset`!!!
2034
2035	This function returns the `VkResult` from `vkFlushMappedMemoryRanges` if it is
2036	called, otherwise `VK_SUCCESS`.
2037	*/
2038	VMA_CALL_PRE VkResult VMA_CALL_POST vmaFlushAllocation(
2039	VmaAllocator VMA_NOT_NULL allocator,
2040	VmaAllocation VMA_NOT_NULL allocation,
2041	VkDeviceSize offset,
2042	VkDeviceSize size);
2043
2044	/* \brief Invalidates memory of given allocation.*
2045
2046	Calls `vkInvalidateMappedMemoryRanges()` for memory associated with given range of given allocation.
2047	It needs to be called before reading from a mapped memory for memory types that are not `HOST_COHERENT`.
2048	Map operation doesn't do that automatically.
2049
2050	- `offset` must be relative to the beginning of allocation.
2051	- `size` can be `VK_WHOLE_SIZE`. It means all memory from `offset` the the end of given allocation.
2052	- `offset` and `size` don't have to be aligned.
2053	They are internally rounded down/up to multiply of `nonCoherentAtomSize`.
2054	- If `size` is 0, this call is ignored.
2055	- If memory type that the `allocation` belongs to is not `HOST_VISIBLE` or it is `HOST_COHERENT`,
2056	this call is ignored.
2057
2058	Warning! `offset` and `size` are relative to the contents of given `allocation`.
2059	If you mean whole allocation, you can pass 0 and `VK_WHOLE_SIZE`, respectively.
2060	Do not pass allocation's offset as `offset`!!!
2061
2062	This function returns the `VkResult` from `vkInvalidateMappedMemoryRanges` if
2063	it is called, otherwise `VK_SUCCESS`.
2064	*/
2065	VMA_CALL_PRE VkResult VMA_CALL_POST vmaInvalidateAllocation(
2066	VmaAllocator VMA_NOT_NULL allocator,
2067	VmaAllocation VMA_NOT_NULL allocation,
2068	VkDeviceSize offset,
2069	VkDeviceSize size);
2070
2071	/* \brief Flushes memory of given set of allocations.*
2072
2073	Calls `vkFlushMappedMemoryRanges()` for memory associated with given ranges of given allocations.
2074	For more information, see documentation of vmaFlushAllocation().
2075
2076	\param allocator
2077	\param allocationCount
2078	\param allocations
2079	\param offsets If not null, it must point to an array of offsets of regions to flush, relative to the beginning of respective allocations. Null means all ofsets are zero.
2080	\param sizes If not null, it must point to an array of sizes of regions to flush in respective allocations. Null means `VK_WHOLE_SIZE` for all allocations.
2081
2082	This function returns the `VkResult` from `vkFlushMappedMemoryRanges` if it is
2083	called, otherwise `VK_SUCCESS`.
2084	*/
2085	VMA_CALL_PRE VkResult VMA_CALL_POST vmaFlushAllocations(
2086	VmaAllocator VMA_NOT_NULL allocator,
2087	uint32_t allocationCount,
2088	const VmaAllocation VMA_NOT_NULL* VMA_NULLABLE VMA_LEN_IF_NOT_NULL(allocationCount) allocations,
2089	const VkDeviceSize* VMA_NULLABLE VMA_LEN_IF_NOT_NULL(allocationCount) offsets,
2090	const VkDeviceSize* VMA_NULLABLE VMA_LEN_IF_NOT_NULL(allocationCount) sizes);
2091
2092	/* \brief Invalidates memory of given set of allocations.*
2093
2094	Calls `vkInvalidateMappedMemoryRanges()` for memory associated with given ranges of given allocations.
2095	For more information, see documentation of vmaInvalidateAllocation().
2096
2097	\param allocator
2098	\param allocationCount
2099	\param allocations
2100	\param offsets If not null, it must point to an array of offsets of regions to flush, relative to the beginning of respective allocations. Null means all ofsets are zero.
2101	\param sizes If not null, it must point to an array of sizes of regions to flush in respective allocations. Null means `VK_WHOLE_SIZE` for all allocations.
2102
2103	This function returns the `VkResult` from `vkInvalidateMappedMemoryRanges` if it is
2104	called, otherwise `VK_SUCCESS`.
2105	*/
2106	VMA_CALL_PRE VkResult VMA_CALL_POST vmaInvalidateAllocations(
2107	VmaAllocator VMA_NOT_NULL allocator,
2108	uint32_t allocationCount,
2109	const VmaAllocation VMA_NOT_NULL* VMA_NULLABLE VMA_LEN_IF_NOT_NULL(allocationCount) allocations,
2110	const VkDeviceSize* VMA_NULLABLE VMA_LEN_IF_NOT_NULL(allocationCount) offsets,
2111	const VkDeviceSize* VMA_NULLABLE VMA_LEN_IF_NOT_NULL(allocationCount) sizes);
2112
2113	/* \brief Checks magic number in margins around all allocations in given memory types (in both default and custom pools) in search for corruptions.*
2114
2115	\param allocator
2116	\param memoryTypeBits Bit mask, where each bit set means that a memory type with that index should be checked.
2117
2118	Corruption detection is enabled only when `VMA_DEBUG_DETECT_CORRUPTION` macro is defined to nonzero,
2119	`VMA_DEBUG_MARGIN` is defined to nonzero and only for memory types that are
2120	`HOST_VISIBLE` and `HOST_COHERENT`. For more information, see [Corruption detection](@ref debugging_memory_usage_corruption_detection).
2121
2122	Possible return values:
2123
2124	- `VK_ERROR_FEATURE_NOT_PRESENT` - corruption detection is not enabled for any of specified memory types.
2125	- `VK_SUCCESS` - corruption detection has been performed and succeeded.
2126	- `VK_ERROR_UNKNOWN` - corruption detection has been performed and found memory corruptions around one of the allocations.
2127	`VMA_ASSERT` is also fired in that case.
2128	- Other value: Error returned by Vulkan, e.g. memory mapping failure.
2129	*/
2130	VMA_CALL_PRE VkResult VMA_CALL_POST vmaCheckCorruption(
2131	VmaAllocator VMA_NOT_NULL allocator,
2132	uint32_t memoryTypeBits);
2133
2134	/* \brief Begins defragmentation process.*
2135
2136	\param allocator Allocator object.
2137	\param pInfo Structure filled with parameters of defragmentation.
2138	\param[out] pContext Context object that must be passed to vmaEndDefragmentation() to finish defragmentation.
2139	\returns
2140	- `VK_SUCCESS` if defragmentation can begin.
2141	- `VK_ERROR_FEATURE_NOT_PRESENT` if defragmentation is not supported.
2142
2143	For more information about defragmentation, see documentation chapter:
2144	[Defragmentation](@ref defragmentation).
2145	*/
2146	VMA_CALL_PRE VkResult VMA_CALL_POST vmaBeginDefragmentation(
2147	VmaAllocator VMA_NOT_NULL allocator,
2148	const VmaDefragmentationInfo* VMA_NOT_NULL pInfo,
2149	VmaDefragmentationContext VMA_NULLABLE* VMA_NOT_NULL pContext);
2150
2151	/* \brief Ends defragmentation process.*
2152
2153	\param allocator Allocator object.
2154	\param context Context object that has been created by vmaBeginDefragmentation().
2155	\param[out] pStats Optional stats for the defragmentation. Can be null.
2156
2157	Use this function to finish defragmentation started by vmaBeginDefragmentation().
2158	*/
2159	VMA_CALL_PRE void VMA_CALL_POST vmaEndDefragmentation(
2160	VmaAllocator VMA_NOT_NULL allocator,
2161	VmaDefragmentationContext VMA_NOT_NULL context,
2162	VmaDefragmentationStats* VMA_NULLABLE pStats);
2163
2164	/* \brief Starts single defragmentation pass.*
2165
2166	\param allocator Allocator object.
2167	\param context Context object that has been created by vmaBeginDefragmentation().
2168	\param[out] pPassInfo Computed informations for current pass.
2169	\returns
2170	- `VK_SUCCESS` if no more moves are possible. Then you can omit call to vmaEndDefragmentationPass() and simply end whole defragmentation.
2171	- `VK_INCOMPLETE` if there are pending moves returned in `pPassInfo`. You need to perform them, call vmaEndDefragmentationPass(),
2172	and then preferably try another pass with vmaBeginDefragmentationPass().
2173	*/
2174	VMA_CALL_PRE VkResult VMA_CALL_POST vmaBeginDefragmentationPass(
2175	VmaAllocator VMA_NOT_NULL allocator,
2176	VmaDefragmentationContext VMA_NOT_NULL context,
2177	VmaDefragmentationPassMoveInfo* VMA_NOT_NULL pPassInfo);
2178
2179	/* \brief Ends single defragmentation pass.*
2180
2181	\param allocator Allocator object.
2182	\param context Context object that has been created by vmaBeginDefragmentation().
2183	\param pPassInfo Computed informations for current pass filled by vmaBeginDefragmentationPass() and possibly modified by you.
2184
2185	Returns `VK_SUCCESS` if no more moves are possible or `VK_INCOMPLETE` if more defragmentations are possible.
2186
2187	Ends incremental defragmentation pass and commits all defragmentation moves from `pPassInfo`.
2188	After this call:
2189
2190	- Allocations at `pPassInfo[i].srcAllocation` that had `pPassInfo[i].operation ==` #VMA_DEFRAGMENTATION_MOVE_OPERATION_COPY
2191	(which is the default) will be pointing to the new destination place.
2192	- Allocation at `pPassInfo[i].srcAllocation` that had `pPassInfo[i].operation ==` #VMA_DEFRAGMENTATION_MOVE_OPERATION_DESTROY
2193	will be freed.
2194
2195	If no more moves are possible you can end whole defragmentation.
2196	*/
2197	VMA_CALL_PRE VkResult VMA_CALL_POST vmaEndDefragmentationPass(
2198	VmaAllocator VMA_NOT_NULL allocator,
2199	VmaDefragmentationContext VMA_NOT_NULL context,
2200	VmaDefragmentationPassMoveInfo* VMA_NOT_NULL pPassInfo);
2201
2202	/* \brief Binds buffer to allocation.*
2203
2204	Binds specified buffer to region of memory represented by specified allocation.
2205	Gets `VkDeviceMemory` handle and offset from the allocation.
2206	If you want to create a buffer, allocate memory for it and bind them together separately,
2207	you should use this function for binding instead of standard `vkBindBufferMemory()`,
2208	because it ensures proper synchronization so that when a `VkDeviceMemory` object is used by multiple
2209	allocations, calls to `vkBindMemory()` or `vkMapMemory()` won't happen from multiple threads simultaneously*
2210	(which is illegal in Vulkan).
2211
2212	It is recommended to use function vmaCreateBuffer() instead of this one.
2213	*/
2214	VMA_CALL_PRE VkResult VMA_CALL_POST vmaBindBufferMemory(
2215	VmaAllocator VMA_NOT_NULL allocator,
2216	VmaAllocation VMA_NOT_NULL allocation,
2217	VkBuffer VMA_NOT_NULL_NON_DISPATCHABLE buffer);
2218
2219	/* \brief Binds buffer to allocation with additional parameters.*
2220
2221	\param allocator
2222	\param allocation
2223	\param allocationLocalOffset Additional offset to be added while binding, relative to the beginning of the `allocation`. Normally it should be 0.
2224	\param buffer
2225	\param pNext A chain of structures to be attached to `VkBindBufferMemoryInfoKHR` structure used internally. Normally it should be null.
2226
2227	This function is similar to vmaBindBufferMemory(), but it provides additional parameters.
2228
2229	If `pNext` is not null, #VmaAllocator object must have been created with #VMA_ALLOCATOR_CREATE_KHR_BIND_MEMORY2_BIT flag
2230	or with VmaAllocatorCreateInfo::vulkanApiVersion `>= VK_API_VERSION_1_1`. Otherwise the call fails.
2231	*/
2232	VMA_CALL_PRE VkResult VMA_CALL_POST vmaBindBufferMemory2(
2233	VmaAllocator VMA_NOT_NULL allocator,
2234	VmaAllocation VMA_NOT_NULL allocation,
2235	VkDeviceSize allocationLocalOffset,
2236	VkBuffer VMA_NOT_NULL_NON_DISPATCHABLE buffer,
2237	const void* VMA_NULLABLE pNext);
2238
2239	/* \brief Binds image to allocation.*
2240
2241	Binds specified image to region of memory represented by specified allocation.
2242	Gets `VkDeviceMemory` handle and offset from the allocation.
2243	If you want to create an image, allocate memory for it and bind them together separately,
2244	you should use this function for binding instead of standard `vkBindImageMemory()`,
2245	because it ensures proper synchronization so that when a `VkDeviceMemory` object is used by multiple
2246	allocations, calls to `vkBindMemory()` or `vkMapMemory()` won't happen from multiple threads simultaneously*
2247	(which is illegal in Vulkan).
2248
2249	It is recommended to use function vmaCreateImage() instead of this one.
2250	*/
2251	VMA_CALL_PRE VkResult VMA_CALL_POST vmaBindImageMemory(
2252	VmaAllocator VMA_NOT_NULL allocator,
2253	VmaAllocation VMA_NOT_NULL allocation,
2254	VkImage VMA_NOT_NULL_NON_DISPATCHABLE image);
2255
2256	/* \brief Binds image to allocation with additional parameters.*
2257
2258	\param allocator
2259	\param allocation
2260	\param allocationLocalOffset Additional offset to be added while binding, relative to the beginning of the `allocation`. Normally it should be 0.
2261	\param image
2262	\param pNext A chain of structures to be attached to `VkBindImageMemoryInfoKHR` structure used internally. Normally it should be null.
2263
2264	This function is similar to vmaBindImageMemory(), but it provides additional parameters.
2265
2266	If `pNext` is not null, #VmaAllocator object must have been created with #VMA_ALLOCATOR_CREATE_KHR_BIND_MEMORY2_BIT flag
2267	or with VmaAllocatorCreateInfo::vulkanApiVersion `>= VK_API_VERSION_1_1`. Otherwise the call fails.
2268	*/
2269	VMA_CALL_PRE VkResult VMA_CALL_POST vmaBindImageMemory2(
2270	VmaAllocator VMA_NOT_NULL allocator,
2271	VmaAllocation VMA_NOT_NULL allocation,
2272	VkDeviceSize allocationLocalOffset,
2273	VkImage VMA_NOT_NULL_NON_DISPATCHABLE image,
2274	const void* VMA_NULLABLE pNext);
2275
2276	/* \brief Creates a new `VkBuffer`, allocates and binds memory for it.*
2277
2278	\param allocator
2279	\param pBufferCreateInfo
2280	\param pAllocationCreateInfo
2281	\param[out] pBuffer Buffer that was created.
2282	\param[out] pAllocation Allocation that was created.
2283	\param[out] pAllocationInfo Optional. Information about allocated memory. It can be later fetched using function vmaGetAllocationInfo().
2284
2285	This function automatically:
2286
2287	-# Creates buffer.
2288	-# Allocates appropriate memory for it.
2289	-# Binds the buffer with the memory.
2290
2291	If any of these operations fail, buffer and allocation are not created,
2292	returned value is negative error code, `pBuffer` and `pAllocation` are null.
2293
2294	If the function succeeded, you must destroy both buffer and allocation when you
2295	no longer need them using either convenience function vmaDestroyBuffer() or
2296	separately, using `vkDestroyBuffer()` and vmaFreeMemory().
2297
2298	If #VMA_ALLOCATOR_CREATE_KHR_DEDICATED_ALLOCATION_BIT flag was used,
2299	VK_KHR_dedicated_allocation extension is used internally to query driver whether
2300	it requires or prefers the new buffer to have dedicated allocation. If yes,
2301	and if dedicated allocation is possible
2302	(#VMA_ALLOCATION_CREATE_NEVER_ALLOCATE_BIT is not used), it creates dedicated
2303	allocation for this buffer, just like when using
2304	#VMA_ALLOCATION_CREATE_DEDICATED_MEMORY_BIT.
2305
2306	\note This function creates a new `VkBuffer`. Sub-allocation of parts of one large buffer,
2307	although recommended as a good practice, is out of scope of this library and could be implemented
2308	by the user as a higher-level logic on top of VMA.
2309	*/
2310	VMA_CALL_PRE VkResult VMA_CALL_POST vmaCreateBuffer(
2311	VmaAllocator VMA_NOT_NULL allocator,
2312	const VkBufferCreateInfo* VMA_NOT_NULL pBufferCreateInfo,
2313	const VmaAllocationCreateInfo* VMA_NOT_NULL pAllocationCreateInfo,
2314	VkBuffer VMA_NULLABLE_NON_DISPATCHABLE* VMA_NOT_NULL pBuffer,
2315	VmaAllocation VMA_NULLABLE* VMA_NOT_NULL pAllocation,
2316	VmaAllocationInfo* VMA_NULLABLE pAllocationInfo);
2317
2318	/* \brief Creates a buffer with additional minimum alignment.*
2319
2320	Similar to vmaCreateBuffer() but provides additional parameter `minAlignment` which allows to specify custom,
2321	minimum alignment to be used when placing the buffer inside a larger memory block, which may be needed e.g.
2322	for interop with OpenGL.
2323	*/
2324	VMA_CALL_PRE VkResult VMA_CALL_POST vmaCreateBufferWithAlignment(
2325	VmaAllocator VMA_NOT_NULL allocator,
2326	const VkBufferCreateInfo* VMA_NOT_NULL pBufferCreateInfo,
2327	const VmaAllocationCreateInfo* VMA_NOT_NULL pAllocationCreateInfo,
2328	VkDeviceSize minAlignment,
2329	VkBuffer VMA_NULLABLE_NON_DISPATCHABLE* VMA_NOT_NULL pBuffer,
2330	VmaAllocation VMA_NULLABLE* VMA_NOT_NULL pAllocation,
2331	VmaAllocationInfo* VMA_NULLABLE pAllocationInfo);
2332
2333	/* \brief Creates a new `VkBuffer`, binds already created memory for it.*
2334
2335	\param allocator
2336	\param allocation Allocation that provides memory to be used for binding new buffer to it.
2337	\param pBufferCreateInfo
2338	\param[out] pBuffer Buffer that was created.
2339
2340	This function automatically:
2341
2342	-# Creates buffer.
2343	-# Binds the buffer with the supplied memory.
2344
2345	If any of these operations fail, buffer is not created,
2346	returned value is negative error code and `pBuffer` is null.*
2347
2348	If the function succeeded, you must destroy the buffer when you
2349	no longer need it using `vkDestroyBuffer()`. If you want to also destroy the corresponding
2350	allocation you can use convenience function vmaDestroyBuffer().
2351	*/
2352	VMA_CALL_PRE VkResult VMA_CALL_POST vmaCreateAliasingBuffer(
2353	VmaAllocator VMA_NOT_NULL allocator,
2354	VmaAllocation VMA_NOT_NULL allocation,
2355	const VkBufferCreateInfo* VMA_NOT_NULL pBufferCreateInfo,
2356	VkBuffer VMA_NULLABLE_NON_DISPATCHABLE* VMA_NOT_NULL pBuffer);
2357
2358	/* \brief Destroys Vulkan buffer and frees allocated memory.*
2359
2360	This is just a convenience function equivalent to:
2361
2362	\code
2363	vkDestroyBuffer(device, buffer, allocationCallbacks);
2364	vmaFreeMemory(allocator, allocation);
2365	\endcode
2366
2367	It is safe to pass null as buffer and/or allocation.
2368	*/
2369	VMA_CALL_PRE void VMA_CALL_POST vmaDestroyBuffer(
2370	VmaAllocator VMA_NOT_NULL allocator,
2371	VkBuffer VMA_NULLABLE_NON_DISPATCHABLE buffer,
2372	VmaAllocation VMA_NULLABLE allocation);
2373
2374	/// Function similar to vmaCreateBuffer().
2375	VMA_CALL_PRE VkResult VMA_CALL_POST vmaCreateImage(
2376	VmaAllocator VMA_NOT_NULL allocator,
2377	const VkImageCreateInfo* VMA_NOT_NULL pImageCreateInfo,
2378	const VmaAllocationCreateInfo* VMA_NOT_NULL pAllocationCreateInfo,
2379	VkImage VMA_NULLABLE_NON_DISPATCHABLE* VMA_NOT_NULL pImage,
2380	VmaAllocation VMA_NULLABLE* VMA_NOT_NULL pAllocation,
2381	VmaAllocationInfo* VMA_NULLABLE pAllocationInfo);
2382
2383	/// Function similar to vmaCreateAliasingBuffer().
2384	VMA_CALL_PRE VkResult VMA_CALL_POST vmaCreateAliasingImage(
2385	VmaAllocator VMA_NOT_NULL allocator,
2386	VmaAllocation VMA_NOT_NULL allocation,
2387	const VkImageCreateInfo* VMA_NOT_NULL pImageCreateInfo,
2388	VkImage VMA_NULLABLE_NON_DISPATCHABLE* VMA_NOT_NULL pImage);
2389
2390	/* \brief Destroys Vulkan image and frees allocated memory.*
2391
2392	This is just a convenience function equivalent to:
2393
2394	\code
2395	vkDestroyImage(device, image, allocationCallbacks);
2396	vmaFreeMemory(allocator, allocation);
2397	\endcode
2398
2399	It is safe to pass null as image and/or allocation.
2400	*/
2401	VMA_CALL_PRE void VMA_CALL_POST vmaDestroyImage(
2402	VmaAllocator VMA_NOT_NULL allocator,
2403	VkImage VMA_NULLABLE_NON_DISPATCHABLE image,
2404	VmaAllocation VMA_NULLABLE allocation);
2405
2406	/* @} /
2407
2408	/**
2409	\addtogroup group_virtual
2410	@{
2411	*/
2412
2413	/* \brief Creates new #VmaVirtualBlock object.*
2414
2415	\param pCreateInfo Parameters for creation.
2416	\param[out] pVirtualBlock Returned virtual block object or `VMA_NULL` if creation failed.
2417	*/
2418	VMA_CALL_PRE VkResult VMA_CALL_POST vmaCreateVirtualBlock(
2419	const VmaVirtualBlockCreateInfo* VMA_NOT_NULL pCreateInfo,
2420	VmaVirtualBlock VMA_NULLABLE* VMA_NOT_NULL pVirtualBlock);
2421
2422	/* \brief Destroys #VmaVirtualBlock object.*
2423
2424	Please note that you should consciously handle virtual allocations that could remain unfreed in the block.
2425	You should either free them individually using vmaVirtualFree() or call vmaClearVirtualBlock()
2426	if you are sure this is what you want. If you do neither, an assert is called.
2427
2428	If you keep pointers to some additional metadata associated with your virtual allocations in their `pUserData`,
2429	don't forget to free them.
2430	*/
2431	VMA_CALL_PRE void VMA_CALL_POST vmaDestroyVirtualBlock(
2432	VmaVirtualBlock VMA_NULLABLE virtualBlock);
2433
2434	/* \brief Returns true of the #VmaVirtualBlock is empty - contains 0 virtual allocations and has all its space available for new allocations.*
2435	*/
2436	VMA_CALL_PRE VkBool32 VMA_CALL_POST vmaIsVirtualBlockEmpty(
2437	VmaVirtualBlock VMA_NOT_NULL virtualBlock);
2438
2439	/* \brief Returns information about a specific virtual allocation within a virtual block, like its size and `pUserData` pointer.*
2440	*/
2441	VMA_CALL_PRE void VMA_CALL_POST vmaGetVirtualAllocationInfo(
2442	VmaVirtualBlock VMA_NOT_NULL virtualBlock,
2443	VmaVirtualAllocation VMA_NOT_NULL_NON_DISPATCHABLE allocation, VmaVirtualAllocationInfo* VMA_NOT_NULL pVirtualAllocInfo);
2444
2445	/* \brief Allocates new virtual allocation inside given #VmaVirtualBlock.*
2446
2447	If the allocation fails due to not enough free space available, `VK_ERROR_OUT_OF_DEVICE_MEMORY` is returned
2448	(despite the function doesn't ever allocate actual GPU memory).
2449	`pAllocation` is then set to `VK_NULL_HANDLE` and `pOffset`, if not null, it set to `UINT64_MAX`.
2450
2451	\param virtualBlock Virtual block
2452	\param pCreateInfo Parameters for the allocation
2453	\param[out] pAllocation Returned handle of the new allocation
2454	\param[out] pOffset Returned offset of the new allocation. Optional, can be null.
2455	*/
2456	VMA_CALL_PRE VkResult VMA_CALL_POST vmaVirtualAllocate(
2457	VmaVirtualBlock VMA_NOT_NULL virtualBlock,
2458	const VmaVirtualAllocationCreateInfo* VMA_NOT_NULL pCreateInfo,
2459	VmaVirtualAllocation VMA_NULLABLE_NON_DISPATCHABLE* VMA_NOT_NULL pAllocation,
2460	VkDeviceSize* VMA_NULLABLE pOffset);
2461
2462	/* \brief Frees virtual allocation inside given #VmaVirtualBlock.*
2463
2464	It is correct to call this function with `allocation == VK_NULL_HANDLE` - it does nothing.
2465	*/
2466	VMA_CALL_PRE void VMA_CALL_POST vmaVirtualFree(
2467	VmaVirtualBlock VMA_NOT_NULL virtualBlock,
2468	VmaVirtualAllocation VMA_NULLABLE_NON_DISPATCHABLE allocation);
2469
2470	/* \brief Frees all virtual allocations inside given #VmaVirtualBlock.*
2471
2472	You must either call this function or free each virtual allocation individually with vmaVirtualFree()
2473	before destroying a virtual block. Otherwise, an assert is called.
2474
2475	If you keep pointer to some additional metadata associated with your virtual allocation in its `pUserData`,
2476	don't forget to free it as well.
2477	*/
2478	VMA_CALL_PRE void VMA_CALL_POST vmaClearVirtualBlock(
2479	VmaVirtualBlock VMA_NOT_NULL virtualBlock);
2480
2481	/* \brief Changes custom pointer associated with given virtual allocation.*
2482	*/
2483	VMA_CALL_PRE void VMA_CALL_POST vmaSetVirtualAllocationUserData(
2484	VmaVirtualBlock VMA_NOT_NULL virtualBlock,
2485	VmaVirtualAllocation VMA_NOT_NULL_NON_DISPATCHABLE allocation,
2486	void* VMA_NULLABLE pUserData);
2487
2488	/* \brief Calculates and returns statistics about virtual allocations and memory usage in given #VmaVirtualBlock.*
2489
2490	This function is fast to call. For more detailed statistics, see vmaCalculateVirtualBlockStatistics().
2491	*/
2492	VMA_CALL_PRE void VMA_CALL_POST vmaGetVirtualBlockStatistics(
2493	VmaVirtualBlock VMA_NOT_NULL virtualBlock,
2494	VmaStatistics* VMA_NOT_NULL pStats);
2495
2496	/* \brief Calculates and returns detailed statistics about virtual allocations and memory usage in given #VmaVirtualBlock.*
2497
2498	This function is slow to call. Use for debugging purposes.
2499	For less detailed statistics, see vmaGetVirtualBlockStatistics().
2500	*/
2501	VMA_CALL_PRE void VMA_CALL_POST vmaCalculateVirtualBlockStatistics(
2502	VmaVirtualBlock VMA_NOT_NULL virtualBlock,
2503	VmaDetailedStatistics* VMA_NOT_NULL pStats);
2504
2505	/* @} /
2506
2507	#if VMA_STATS_STRING_ENABLED
2508	/**
2509	\addtogroup group_stats
2510	@{
2511	*/
2512
2513	/* \brief Builds and returns a null-terminated string in JSON format with information about given #VmaVirtualBlock.*
2514	\param virtualBlock Virtual block.
2515	\param[out] ppStatsString Returned string.
2516	\param detailedMap Pass `VK_FALSE` to only obtain statistics as returned by vmaCalculateVirtualBlockStatistics(). Pass `VK_TRUE` to also obtain full list of allocations and free spaces.
2517
2518	Returned string must be freed using vmaFreeVirtualBlockStatsString().
2519	*/
2520	VMA_CALL_PRE void VMA_CALL_POST vmaBuildVirtualBlockStatsString(
2521	VmaVirtualBlock VMA_NOT_NULL virtualBlock,
2522	char* VMA_NULLABLE* VMA_NOT_NULL ppStatsString,
2523	VkBool32 detailedMap);
2524
2525	/// Frees a string returned by vmaBuildVirtualBlockStatsString().
2526	VMA_CALL_PRE void VMA_CALL_POST vmaFreeVirtualBlockStatsString(
2527	VmaVirtualBlock VMA_NOT_NULL virtualBlock,
2528	char* VMA_NULLABLE pStatsString);
2529
2530	/* \brief Builds and returns statistics as a null-terminated string in JSON format.*
2531	\param allocator
2532	\param[out] ppStatsString Must be freed using vmaFreeStatsString() function.
2533	\param detailedMap
2534	*/
2535	VMA_CALL_PRE void VMA_CALL_POST vmaBuildStatsString(
2536	VmaAllocator VMA_NOT_NULL allocator,
2537	char* VMA_NULLABLE* VMA_NOT_NULL ppStatsString,
2538	VkBool32 detailedMap);
2539
2540	VMA_CALL_PRE void VMA_CALL_POST vmaFreeStatsString(
2541	VmaAllocator VMA_NOT_NULL allocator,
2542	char* VMA_NULLABLE pStatsString);
2543
2544	/* @} /
2545
2546	#endif // VMA_STATS_STRING_ENABLED
2547
2548	#endif // _VMA_FUNCTION_HEADERS
2549
2550	#ifdef __cplusplus
2551	}
2552	#endif
2553
2554	#endif // AMD_VULKAN_MEMORY_ALLOCATOR_H
2555
2556	////////////////////////////////////////////////////////////////////////////////
2557	////////////////////////////////////////////////////////////////////////////////
2558	//
2559	// IMPLEMENTATION
2560	//
2561	////////////////////////////////////////////////////////////////////////////////
2562	////////////////////////////////////////////////////////////////////////////////
2563
2564	// For Visual Studio IntelliSense.
2565	#if defined(__cplusplus) && defined(__INTELLISENSE__)
2566	#define VMA_IMPLEMENTATION
2567	#endif
2568
2569	#ifdef VMA_IMPLEMENTATION
2570	#undef VMA_IMPLEMENTATION
2571
2572	#include <cstdint>
2573	#include <cstdlib>
2574	#include <cstring>
2575	#include <utility>
2576	#include <type_traits>
2577
2578	#ifdef _MSC_VER
2579	#include <intrin.h> // For functions like __popcnt, _BitScanForward etc.
2580	#endif
2581	#if __cplusplus >= 202002L \|\| _MSVC_LANG >= 202002L // C++20
2582	#include <bit> // For std::popcount
2583	#endif
2584
2585	#if VMA_STATS_STRING_ENABLED
2586	#include <cstdio> // For snprintf
2587	#endif
2588
2589	/*******************************************************************************
2590	CONFIGURATION SECTION
2591
2592	Define some of these macros before each #include of this header or change them
2593	here if you need other then default behavior depending on your environment.
2594	*/
2595	#ifndef _VMA_CONFIGURATION
2596
2597	/*
2598	Define this macro to 1 to make the library fetch pointers to Vulkan functions
2599	internally, like:
2600
2601	vulkanFunctions.vkAllocateMemory = &vkAllocateMemory;
2602	*/
2603	#if !defined(VMA_STATIC_VULKAN_FUNCTIONS) && !defined(VK_NO_PROTOTYPES)
2604	#define VMA_STATIC_VULKAN_FUNCTIONS 1
2605	#endif
2606
2607	/*
2608	Define this macro to 1 to make the library fetch pointers to Vulkan functions
2609	internally, like:
2610
2611	vulkanFunctions.vkAllocateMemory = (PFN_vkAllocateMemory)vkGetDeviceProcAddr(device, "vkAllocateMemory");
2612
2613	To use this feature in new versions of VMA you now have to pass
2614	VmaVulkanFunctions::vkGetInstanceProcAddr and vkGetDeviceProcAddr as
2615	VmaAllocatorCreateInfo::pVulkanFunctions. Other members can be null.
2616	*/
2617	#if !defined(VMA_DYNAMIC_VULKAN_FUNCTIONS)
2618	#define VMA_DYNAMIC_VULKAN_FUNCTIONS 1
2619	#endif
2620
2621	#ifndef VMA_USE_STL_SHARED_MUTEX
2622	// Compiler conforms to C++17.
2623	#if __cplusplus >= 201703L
2624	#define VMA_USE_STL_SHARED_MUTEX 1
2625	// Visual studio defines __cplusplus properly only when passed additional parameter: /Zc:__cplusplus
2626	// Otherwise it is always 199711L, despite shared_mutex works since Visual Studio 2015 Update 2.
2627	#elif defined(_MSC_FULL_VER) && _MSC_FULL_VER >= 190023918 && __cplusplus == 199711L && _MSVC_LANG >= 201703L
2628	#define VMA_USE_STL_SHARED_MUTEX 1
2629	#else
2630	#define VMA_USE_STL_SHARED_MUTEX 0
2631	#endif
2632	#endif
2633
2634	/*
2635	Define this macro to include custom header files without having to edit this file directly, e.g.:
2636
2637	// Inside of "my_vma_configuration_user_includes.h":
2638
2639	#include "my_custom_assert.h" // for MY_CUSTOM_ASSERT
2640	#include "my_custom_min.h" // for my_custom_min
2641	#include <algorithm>
2642	#include <mutex>
2643
2644	// Inside a different file, which includes "vk_mem_alloc.h":
2645
2646	#define VMA_CONFIGURATION_USER_INCLUDES_H "my_vma_configuration_user_includes.h"
2647	#define VMA_ASSERT(expr) MY_CUSTOM_ASSERT(expr)
2648	#define VMA_MIN(v1, v2) (my_custom_min(v1, v2))
2649	#include "vk_mem_alloc.h"
2650	...
2651
2652	The following headers are used in this CONFIGURATION section only, so feel free to
2653	remove them if not needed.
2654	*/
2655	#if !defined(VMA_CONFIGURATION_USER_INCLUDES_H)
2656	#include <cassert> // for assert
2657	#include <algorithm> // for min, max
2658	#include <mutex>
2659	#else
2660	#include VMA_CONFIGURATION_USER_INCLUDES_H
2661	#endif
2662
2663	#ifndef VMA_NULL
2664	// Value used as null pointer. Define it to e.g.: nullptr, NULL, 0, (void)0.*
2665	#define VMA_NULL nullptr
2666	#endif
2667
2668	#if defined(__ANDROID_API__) && (__ANDROID_API__ < 16)
2669	#include <cstdlib>
2670	static void* vma_aligned_alloc(size_t alignment, size_t size)
2671	{
2672	// alignment must be >= sizeof(void)*
2673	if(alignment < sizeof(void*))
2674	{
2675	alignment = sizeof(void*);
2676	}
2677
2678	return memalign(alignment, size);
2679	}
2680	#elif defined(__APPLE__) \|\| defined(__ANDROID__) \|\| (defined(__linux__) && defined(__GLIBCXX__) && !defined(_GLIBCXX_HAVE_ALIGNED_ALLOC))
2681	#include <cstdlib>
2682
2683	#if defined(__APPLE__)
2684	#include <AvailabilityMacros.h>
2685	#endif
2686
2687	static void* vma_aligned_alloc(size_t alignment, size_t size)
2688	{
2689	// Unfortunately, aligned_alloc causes VMA to crash due to it returning null pointers. (At least under 11.4)
2690	// Therefore, for now disable this specific exception until a proper solution is found.
2691	//#if defined(__APPLE__) && (defined(MAC_OS_X_VERSION_10_16) \|\| defined(__IPHONE_14_0))
2692	//#if MAC_OS_X_VERSION_MAX_ALLOWED >= MAC_OS_X_VERSION_10_16 \|\| __IPHONE_OS_VERSION_MAX_ALLOWED >= __IPHONE_14_0
2693	// // For C++14, usr/include/malloc/_malloc.h declares aligned_alloc()) only
2694	// // with the MacOSX11.0 SDK in Xcode 12 (which is what adds
2695	// // MAC_OS_X_VERSION_10_16), even though the function is marked
2696	// // availabe for 10.15. That is why the preprocessor checks for 10.16 but
2697	// // the __builtin_available checks for 10.15.
2698	// // People who use C++17 could call aligned_alloc with the 10.15 SDK already.
2699	// if (__builtin_available(macOS 10.15, iOS 13, ))*
2700	// return aligned_alloc(alignment, size);
2701	//#endif
2702	//#endif
2703
2704	// alignment must be >= sizeof(void)*
2705	if(alignment < sizeof(void*))
2706	{
2707	alignment = sizeof(void*);
2708	}
2709
2710	void *pointer;
2711	if(posix_memalign(&pointer, alignment, size) == `0`)
2712	return pointer;
2713	return VMA_NULL;
2714	}
2715	#elif defined(_WIN32)
2716	static void* vma_aligned_alloc(size_t alignment, size_t size)
2717	{
2718	return _aligned_malloc(size, alignment);
2719	}
2720	#else
2721	static void* vma_aligned_alloc(size_t alignment, size_t size)
2722	{
2723	return aligned_alloc(alignment, size);
2724	}
2725	#endif
2726
2727	#if defined(_WIN32)
2728	static void vma_aligned_free(void* ptr)
2729	{
2730	_aligned_free(ptr);
2731	}
2732	#else
2733	static void vma_aligned_free(void* VMA_NULLABLE ptr)
2734	{
2735	free(ptr);
2736	}
2737	#endif
2738
2739	// If your compiler is not compatible with C++11 and definition of
2740	// aligned_alloc() function is missing, uncommeting following line may help:
2741
2742	//#include <malloc.h>
2743
2744	// Normal assert to check for programmer's errors, especially in Debug configuration.
2745	#ifndef VMA_ASSERT
2746	#ifdef NDEBUG
2747	#define VMA_ASSERT(expr)
2748	#else
2749	#define VMA_ASSERT(expr) assert(expr)
2750	#endif
2751	#endif
2752
2753	// Assert that will be called very often, like inside data structures e.g. operator[].
2754	// Making it non-empty can make program slow.
2755	#ifndef VMA_HEAVY_ASSERT
2756	#ifdef NDEBUG
2757	#define VMA_HEAVY_ASSERT(expr)
2758	#else
2759	#define VMA_HEAVY_ASSERT(expr) //VMA_ASSERT(expr)
2760	#endif
2761	#endif
2762
2763	#ifndef VMA_ALIGN_OF
2764	#define VMA_ALIGN_OF(type) (__alignof(type))
2765	#endif
2766
2767	#ifndef VMA_SYSTEM_ALIGNED_MALLOC
2768	#define VMA_SYSTEM_ALIGNED_MALLOC(size, alignment) vma_aligned_alloc((alignment), (size))
2769	#endif
2770
2771	#ifndef VMA_SYSTEM_ALIGNED_FREE
2772	// VMA_SYSTEM_FREE is the old name, but might have been defined by the user
2773	#if defined(VMA_SYSTEM_FREE)
2774	#define VMA_SYSTEM_ALIGNED_FREE(ptr) VMA_SYSTEM_FREE(ptr)
2775	#else
2776	#define VMA_SYSTEM_ALIGNED_FREE(ptr) vma_aligned_free(ptr)
2777	#endif
2778	#endif
2779
2780	#ifndef VMA_COUNT_BITS_SET
2781	// Returns number of bits set to 1 in (v)
2782	#define VMA_COUNT_BITS_SET(v) VmaCountBitsSet(v)
2783	#endif
2784
2785	#ifndef VMA_BITSCAN_LSB
2786	// Scans integer for index of first nonzero value from the Least Significant Bit (LSB). If mask is 0 then returns UINT8_MAX
2787	#define VMA_BITSCAN_LSB(mask) VmaBitScanLSB(mask)
2788	#endif
2789
2790	#ifndef VMA_BITSCAN_MSB
2791	// Scans integer for index of first nonzero value from the Most Significant Bit (MSB). If mask is 0 then returns UINT8_MAX
2792	#define VMA_BITSCAN_MSB(mask) VmaBitScanMSB(mask)
2793	#endif
2794
2795	#ifndef VMA_MIN
2796	#define VMA_MIN(v1, v2) ((std::min)((v1), (v2)))
2797	#endif
2798
2799	#ifndef VMA_MAX
2800	#define VMA_MAX(v1, v2) ((std::max)((v1), (v2)))
2801	#endif
2802
2803	#ifndef VMA_SWAP
2804	#define VMA_SWAP(v1, v2) std::swap((v1), (v2))
2805	#endif
2806
2807	#ifndef VMA_SORT
2808	#define VMA_SORT(beg, end, cmp) std::sort(beg, end, cmp)
2809	#endif
2810
2811	#ifndef VMA_DEBUG_LOG
2812	#define VMA_DEBUG_LOG(format, ...)
2813	/*
2814	#define VMA_DEBUG_LOG(format, ...) do { \
2815	printf(format, __VA_ARGS__); \
2816	printf("\n"); \
2817	} while(false)
2818	*/
2819	#endif
2820
2821	// Define this macro to 1 to enable functions: vmaBuildStatsString, vmaFreeStatsString.
2822	#if VMA_STATS_STRING_ENABLED
2823	static inline void VmaUint32ToStr(char* VMA_NOT_NULL outStr, size_t strLen, uint32_t num)
2824	{
2825	snprintf(outStr, strLen, "%u", static_cast<unsigned int>(num));
2826	}
2827	static inline void VmaUint64ToStr(char* VMA_NOT_NULL outStr, size_t strLen, uint64_t num)
2828	{
2829	snprintf(outStr, strLen, "%llu", static_cast<unsigned long long>(num));
2830	}
2831	static inline void VmaPtrToStr(char* VMA_NOT_NULL outStr, size_t strLen, const void* ptr)
2832	{
2833	snprintf(outStr, strLen, "%p", ptr);
2834	}
2835	#endif
2836
2837	#ifndef VMA_MUTEX
2838	class VmaMutex
2839	{
2840	public:
2841	void Lock() { m_Mutex.lock(); }
2842	void Unlock() { m_Mutex.unlock(); }
2843	bool TryLock() { return m_Mutex.try_lock(); }
2844	private:
2845	std::mutex m_Mutex;
2846	};
2847	#define VMA_MUTEX VmaMutex
2848	#endif
2849
2850	// Read-write mutex, where "read" is shared access, "write" is exclusive access.
2851	#ifndef VMA_RW_MUTEX
2852	#if VMA_USE_STL_SHARED_MUTEX
2853	// Use std::shared_mutex from C++17.
2854	#include <shared_mutex>
2855	class VmaRWMutex
2856	{
2857	public:
2858	void LockRead() { m_Mutex.lock_shared(); }
2859	void UnlockRead() { m_Mutex.unlock_shared(); }
2860	bool TryLockRead() { return m_Mutex.try_lock_shared(); }
2861	void LockWrite() { m_Mutex.lock(); }
2862	void UnlockWrite() { m_Mutex.unlock(); }
2863	bool TryLockWrite() { return m_Mutex.try_lock(); }
2864	private:
2865	std::shared_mutex m_Mutex;
2866	};
2867	#define VMA_RW_MUTEX VmaRWMutex
2868	#elif defined(_WIN32) && defined(WINVER) && WINVER >= 0x0600
2869	// Use SRWLOCK from WinAPI.
2870	// Minimum supported client = Windows Vista, server = Windows Server 2008.
2871	class VmaRWMutex
2872	{
2873	public:
2874	VmaRWMutex() { InitializeSRWLock(&m_Lock); }
2875	void LockRead() { AcquireSRWLockShared(&m_Lock); }
2876	void UnlockRead() { ReleaseSRWLockShared(&m_Lock); }
2877	bool TryLockRead() { return TryAcquireSRWLockShared(&m_Lock) != FALSE; }
2878	void LockWrite() { AcquireSRWLockExclusive(&m_Lock); }
2879	void UnlockWrite() { ReleaseSRWLockExclusive(&m_Lock); }
2880	bool TryLockWrite() { return TryAcquireSRWLockExclusive(&m_Lock) != FALSE; }
2881	private:
2882	SRWLOCK m_Lock;
2883	};
2884	#define VMA_RW_MUTEX VmaRWMutex
2885	#else
2886	// Less efficient fallback: Use normal mutex.
2887	class VmaRWMutex
2888	{
2889	public:
2890	void LockRead() { m_Mutex.Lock(); }
2891	void UnlockRead() { m_Mutex.Unlock(); }
2892	bool TryLockRead() { return m_Mutex.TryLock(); }
2893	void LockWrite() { m_Mutex.Lock(); }
2894	void UnlockWrite() { m_Mutex.Unlock(); }
2895	bool TryLockWrite() { return m_Mutex.TryLock(); }
2896	private:
2897	VMA_MUTEX m_Mutex;
2898	};
2899	#define VMA_RW_MUTEX VmaRWMutex
2900	#endif // #if VMA_USE_STL_SHARED_MUTEX
2901	#endif // #ifndef VMA_RW_MUTEX
2902
2903	/*
2904	If providing your own implementation, you need to implement a subset of std::atomic.
2905	*/
2906	#ifndef VMA_ATOMIC_UINT32
2907	#include <atomic>
2908	#define VMA_ATOMIC_UINT32 std::atomic<uint32_t>
2909	#endif
2910
2911	#ifndef VMA_ATOMIC_UINT64
2912	#include <atomic>
2913	#define VMA_ATOMIC_UINT64 std::atomic<uint64_t>
2914	#endif
2915
2916	#ifndef VMA_DEBUG_ALWAYS_DEDICATED_MEMORY
2917	/**
2918	Every allocation will have its own memory block.
2919	Define to 1 for debugging purposes only.
2920	*/
2921	#define VMA_DEBUG_ALWAYS_DEDICATED_MEMORY (0)
2922	#endif
2923
2924	#ifndef VMA_MIN_ALIGNMENT
2925	/**
2926	Minimum alignment of all allocations, in bytes.
2927	Set to more than 1 for debugging purposes. Must be power of two.
2928	*/
2929	#ifdef VMA_DEBUG_ALIGNMENT // Old name
2930	#define VMA_MIN_ALIGNMENT VMA_DEBUG_ALIGNMENT
2931	#else
2932	#define VMA_MIN_ALIGNMENT (1)
2933	#endif
2934	#endif
2935
2936	#ifndef VMA_DEBUG_MARGIN
2937	/**
2938	Minimum margin after every allocation, in bytes.
2939	Set nonzero for debugging purposes only.
2940	*/
2941	#define VMA_DEBUG_MARGIN (0)
2942	#endif
2943
2944	#ifndef VMA_DEBUG_INITIALIZE_ALLOCATIONS
2945	/**
2946	Define this macro to 1 to automatically fill new allocations and destroyed
2947	allocations with some bit pattern.
2948	*/
2949	#define VMA_DEBUG_INITIALIZE_ALLOCATIONS (0)
2950	#endif
2951
2952	#ifndef VMA_DEBUG_DETECT_CORRUPTION
2953	/**
2954	Define this macro to 1 together with non-zero value of VMA_DEBUG_MARGIN to
2955	enable writing magic value to the margin after every allocation and
2956	validating it, so that memory corruptions (out-of-bounds writes) are detected.
2957	*/
2958	#define VMA_DEBUG_DETECT_CORRUPTION (0)
2959	#endif
2960
2961	#ifndef VMA_DEBUG_GLOBAL_MUTEX
2962	/**
2963	Set this to 1 for debugging purposes only, to enable single mutex protecting all
2964	entry calls to the library. Can be useful for debugging multithreading issues.
2965	*/
2966	#define VMA_DEBUG_GLOBAL_MUTEX (0)
2967	#endif
2968
2969	#ifndef VMA_DEBUG_MIN_BUFFER_IMAGE_GRANULARITY
2970	/**
2971	Minimum value for VkPhysicalDeviceLimits::bufferImageGranularity.
2972	Set to more than 1 for debugging purposes only. Must be power of two.
2973	*/
2974	#define VMA_DEBUG_MIN_BUFFER_IMAGE_GRANULARITY (1)
2975	#endif
2976
2977	#ifndef VMA_DEBUG_DONT_EXCEED_MAX_MEMORY_ALLOCATION_COUNT
2978	/*
2979	Set this to 1 to make VMA never exceed VkPhysicalDeviceLimits::maxMemoryAllocationCount
2980	and return error instead of leaving up to Vulkan implementation what to do in such cases.
2981	*/
2982	#define VMA_DEBUG_DONT_EXCEED_MAX_MEMORY_ALLOCATION_COUNT (0)
2983	#endif
2984
2985	#ifndef VMA_SMALL_HEAP_MAX_SIZE
2986	/// Maximum size of a memory heap in Vulkan to consider it "small".
2987	#define VMA_SMALL_HEAP_MAX_SIZE (1024ull * 1024 * 1024)
2988	#endif
2989
2990	#ifndef VMA_DEFAULT_LARGE_HEAP_BLOCK_SIZE
2991	/// Default size of a block allocated as single VkDeviceMemory from a "large" heap.
2992	#define VMA_DEFAULT_LARGE_HEAP_BLOCK_SIZE (256ull * 1024 * 1024)
2993	#endif
2994
2995	/*
2996	Mapping hysteresis is a logic that launches when vmaMapMemory/vmaUnmapMemory is called
2997	or a persistently mapped allocation is created and destroyed several times in a row.
2998	It keeps additional +1 mapping of a device memory block to prevent calling actual
2999	vkMapMemory/vkUnmapMemory too many times, which may improve performance and help
3000	tools like RenderDOc.
3001	*/
3002	#ifndef VMA_MAPPING_HYSTERESIS_ENABLED
3003	#define VMA_MAPPING_HYSTERESIS_ENABLED 1
3004	#endif
3005
3006	#ifndef VMA_CLASS_NO_COPY
3007	#define VMA_CLASS_NO_COPY(className) \
3008	private: \
3009	className(const className&) = delete; \
3010	className& operator=(const className&) = delete;
3011	#endif
3012
3013	#define VMA_VALIDATE(cond) do { if(!(cond)) { \
3014	VMA_ASSERT(0 && "Validation failed: " #cond); \
3015	return false; \
3016	} } while(false)
3017
3018	/*******************************************************************************
3019	END OF CONFIGURATION
3020	*/
3021	#endif // _VMA_CONFIGURATION
3022
3023
3024	static const uint8_t VMA_ALLOCATION_FILL_PATTERN_CREATED = `0xDC`;
3025	static const uint8_t VMA_ALLOCATION_FILL_PATTERN_DESTROYED = `0xEF`;
3026	// Decimal 2139416166, float NaN, little-endian binary 66 E6 84 7F.
3027	static const uint32_t VMA_CORRUPTION_DETECTION_MAGIC_VALUE = `0x7F84E666`;
3028
3029	// Copy of some Vulkan definitions so we don't need to check their existence just to handle few constants.
3030	static const uint32_t VK_MEMORY_PROPERTY_DEVICE_COHERENT_BIT_AMD_COPY = `0x00000040`;
3031	static const uint32_t VK_MEMORY_PROPERTY_DEVICE_UNCACHED_BIT_AMD_COPY = `0x00000080`;
3032	static const uint32_t VK_BUFFER_USAGE_SHADER_DEVICE_ADDRESS_BIT_COPY = `0x00020000`;
3033	static const uint32_t VK_IMAGE_CREATE_DISJOINT_BIT_COPY = `0x00000200`;
3034	static const int32_t VK_IMAGE_TILING_DRM_FORMAT_MODIFIER_EXT_COPY = `1000158000`;
3035	static const uint32_t VMA_ALLOCATION_INTERNAL_STRATEGY_MIN_OFFSET = `0x10000000u`;
3036	static const uint32_t VMA_ALLOCATION_TRY_COUNT = `32`;
3037	static const uint32_t VMA_VENDOR_ID_AMD = `4098`;
3038
3039	// This one is tricky. Vulkan specification defines this code as available since
3040	// Vulkan 1.0, but doesn't actually define it in Vulkan SDK earlier than 1.2.131.
3041	// See pull request #207.
3042	#define VK_ERROR_UNKNOWN_COPY ((VkResult)-13)
3043
3044
3045	#if VMA_STATS_STRING_ENABLED
3046	// Correspond to values of enum VmaSuballocationType.
3047	static const char* VMA_SUBALLOCATION_TYPE_NAMES[] =
3048	{
3049	"FREE",
3050	"UNKNOWN",
3051	"BUFFER",
3052	"IMAGE_UNKNOWN",
3053	"IMAGE_LINEAR",
3054	"IMAGE_OPTIMAL",
3055	};
3056	#endif
3057
3058	static VkAllocationCallbacks VmaEmptyAllocationCallbacks =
3059	{ VMA_NULL, VMA_NULL, VMA_NULL, VMA_NULL, VMA_NULL, VMA_NULL };
3060
3061
3062	#ifndef _VMA_ENUM_DECLARATIONS
3063
3064	enum VmaSuballocationType
3065	{
3066	VMA_SUBALLOCATION_TYPE_FREE = `0`,
3067	VMA_SUBALLOCATION_TYPE_UNKNOWN = `1`,
3068	VMA_SUBALLOCATION_TYPE_BUFFER = `2`,
3069	VMA_SUBALLOCATION_TYPE_IMAGE_UNKNOWN = `3`,
3070	VMA_SUBALLOCATION_TYPE_IMAGE_LINEAR = `4`,
3071	VMA_SUBALLOCATION_TYPE_IMAGE_OPTIMAL = `5`,
3072	VMA_SUBALLOCATION_TYPE_MAX_ENUM = `0x7FFFFFFF`
3073	};
3074
3075	enum VMA_CACHE_OPERATION
3076	{
3077	VMA_CACHE_FLUSH,
3078	VMA_CACHE_INVALIDATE
3079	};
3080
3081	enum class VmaAllocationRequestType
3082	{
3083	Normal,
3084	TLSF,
3085	// Used by "Linear" algorithm.
3086	UpperAddress,
3087	EndOf1st,
3088	EndOf2nd,
3089	};
3090
3091	#endif // _VMA_ENUM_DECLARATIONS
3092
3093	#ifndef _VMA_FORWARD_DECLARATIONS
3094	// Opaque handle used by allocation algorithms to identify single allocation in any conforming way.
3095	VK_DEFINE_NON_DISPATCHABLE_HANDLE(VmaAllocHandle);
3096
3097	struct VmaMutexLock;
3098	struct VmaMutexLockRead;
3099	struct VmaMutexLockWrite;
3100
3101	template<typename T>
3102	struct AtomicTransactionalIncrement;
3103
3104	template<typename T>
3105	struct VmaStlAllocator;
3106
3107	template<typename T, typename AllocatorT>
3108	class VmaVector;
3109
3110	template<typename T, typename AllocatorT, size_t N>
3111	class VmaSmallVector;
3112
3113	template<typename T>
3114	class VmaPoolAllocator;
3115
3116	template<typename T>
3117	struct VmaListItem;
3118
3119	template<typename T>
3120	class VmaRawList;
3121
3122	template<typename T, typename AllocatorT>
3123	class VmaList;
3124
3125	template<typename ItemTypeTraits>
3126	class VmaIntrusiveLinkedList;
3127
3128	// Unused in this version
3129	#if 0
3130	template<typename T1, typename T2>
3131	struct VmaPair;
3132	template<typename FirstT, typename SecondT>
3133	struct VmaPairFirstLess;
3134
3135	template<typename KeyT, typename ValueT>
3136	class VmaMap;
3137	#endif
3138
3139	#if VMA_STATS_STRING_ENABLED
3140	class VmaStringBuilder;
3141	class VmaJsonWriter;
3142	#endif
3143
3144	class VmaDeviceMemoryBlock;
3145
3146	struct VmaDedicatedAllocationListItemTraits;
3147	class VmaDedicatedAllocationList;
3148
3149	struct VmaSuballocation;
3150	struct VmaSuballocationOffsetLess;
3151	struct VmaSuballocationOffsetGreater;
3152	struct VmaSuballocationItemSizeLess;
3153
3154	typedef VmaList<VmaSuballocation, VmaStlAllocator<VmaSuballocation>> VmaSuballocationList;
3155
3156	struct VmaAllocationRequest;
3157
3158	class VmaBlockMetadata;
3159	class VmaBlockMetadata_Linear;
3160	class VmaBlockMetadata_TLSF;
3161
3162	class VmaBlockVector;
3163
3164	struct VmaPoolListItemTraits;
3165
3166	struct VmaCurrentBudgetData;
3167
3168	class VmaAllocationObjectAllocator;
3169
3170	#endif // _VMA_FORWARD_DECLARATIONS
3171
3172
3173	#ifndef _VMA_FUNCTIONS
3174
3175	/*
3176	Returns number of bits set to 1 in (v).
3177
3178	On specific platforms and compilers you can use instrinsics like:
3179
3180	Visual Studio:
3181	return __popcnt(v);
3182	GCC, Clang:
3183	return static_cast<uint32_t>(__builtin_popcount(v));
3184
3185	Define macro VMA_COUNT_BITS_SET to provide your optimized implementation.
3186	But you need to check in runtime whether user's CPU supports these, as some old processors don't.
3187	*/
3188	static inline uint32_t VmaCountBitsSet(uint32_t v)
3189	{
3190	#if __cplusplus >= 202002L \|\| _MSVC_LANG >= 202002L // C++20
3191	return std::popcount(v);
3192	#else
3193	uint32_t c = v - ((v >> `1`) & `0x55555555`);
3194	c = ((c >> `2`) & `0x33333333`) + (c & `0x33333333`);
3195	c = ((c >> `4`) + c) & `0x0F0F0F0F`;
3196	c = ((c >> `8`) + c) & `0x00FF00FF`;
3197	c = ((c >> `16`) + c) & `0x0000FFFF`;
3198	return c;
3199	#endif
3200	}
3201
3202	static inline uint8_t VmaBitScanLSB(uint64_t mask)
3203	{
3204	#if defined(_MSC_VER) && defined(_WIN64)
3205	unsigned long pos;
3206	if (_BitScanForward64(&pos, mask))
3207	return static_cast<uint8_t>(pos);
3208	return UINT8_MAX;
3209	#elif defined __GNUC__ \|\| defined __clang__
3210	return static_cast<uint8_t>(__builtin_ffsll(mask)) - `1U`;
3211	#else
3212	uint8_t pos = `0`;
3213	uint64_t bit = `1`;
3214	do
3215	{
3216	if (mask & bit)
3217	return pos;
3218	bit <<= `1`;
3219	} while (pos++ < `63`);
3220	return UINT8_MAX;
3221	#endif
3222	}
3223
3224	static inline uint8_t VmaBitScanLSB(uint32_t mask)
3225	{
3226	#ifdef _MSC_VER
3227	unsigned long pos;
3228	if (_BitScanForward(&pos, mask))
3229	return static_cast<uint8_t>(pos);
3230	return UINT8_MAX;
3231	#elif defined __GNUC__ \|\| defined __clang__
3232	return static_cast<uint8_t>(__builtin_ffs(mask)) - `1U`;
3233	#else
3234	uint8_t pos = `0`;
3235	uint32_t bit = `1`;
3236	do
3237	{
3238	if (mask & bit)
3239	return pos;
3240	bit <<= `1`;
3241	} while (pos++ < `31`);
3242	return UINT8_MAX;
3243	#endif
3244	}
3245
3246	static inline uint8_t VmaBitScanMSB(uint64_t mask)
3247	{
3248	#if defined(_MSC_VER) && defined(_WIN64)
3249	unsigned long pos;
3250	if (_BitScanReverse64(&pos, mask))
3251	return static_cast<uint8_t>(pos);
3252	#elif defined __GNUC__ \|\| defined __clang__
3253	if (mask)
3254	return `63` - static_cast<uint8_t>(__builtin_clzll(mask));
3255	#else
3256	uint8_t pos = `63`;
3257	uint64_t bit = `1ULL` << `63`;
3258	do
3259	{
3260	if (mask & bit)
3261	return pos;
3262	bit >>= `1`;
3263	} while (pos-- > `0`);
3264	#endif
3265	return UINT8_MAX;
3266	}
3267
3268	static inline uint8_t VmaBitScanMSB(uint32_t mask)
3269	{
3270	#ifdef _MSC_VER
3271	unsigned long pos;
3272	if (_BitScanReverse(&pos, mask))
3273	return static_cast<uint8_t>(pos);
3274	#elif defined __GNUC__ \|\| defined __clang__
3275	if (mask)
3276	return `31` - static_cast<uint8_t>(__builtin_clz(mask));
3277	#else
3278	uint8_t pos = `31`;
3279	uint32_t bit = `1UL` << `31`;
3280	do
3281	{
3282	if (mask & bit)
3283	return pos;
3284	bit >>= `1`;
3285	} while (pos-- > `0`);
3286	#endif
3287	return UINT8_MAX;
3288	}
3289
3290	/*
3291	Returns true if given number is a power of two.
3292	T must be unsigned integer number or signed integer but always nonnegative.
3293	For 0 returns true.
3294	*/
3295	template <typename T>
3296	inline bool VmaIsPow2(T x)
3297	{
3298	return (x & (x - `1`)) == `0`;
3299	}
3300
3301	// Aligns given value up to nearest multiply of align value. For example: VmaAlignUp(11, 8) = 16.
3302	// Use types like uint32_t, uint64_t as T.
3303	template <typename T>
3304	static inline T VmaAlignUp(T val, T alignment)
3305	{
3306	VMA_HEAVY_ASSERT(VmaIsPow2(alignment));
3307	return (val + alignment - `1`) & ~(alignment - `1`);
3308	}
3309
3310	// Aligns given value down to nearest multiply of align value. For example: VmaAlignUp(11, 8) = 8.
3311	// Use types like uint32_t, uint64_t as T.
3312	template <typename T>
3313	static inline T VmaAlignDown(T val, T alignment)
3314	{
3315	VMA_HEAVY_ASSERT(VmaIsPow2(alignment));
3316	return val & ~(alignment - `1`);
3317	}
3318
3319	// Division with mathematical rounding to nearest number.
3320	template <typename T>
3321	static inline T VmaRoundDiv(T x, T y)
3322	{
3323	return (x + (y / (T)`2`)) / y;
3324	}
3325
3326	// Divide by 'y' and round up to nearest integer.
3327	template <typename T>
3328	static inline T VmaDivideRoundingUp(T x, T y)
3329	{
3330	return (x + y - (T)`1`) / y;
3331	}
3332
3333	// Returns smallest power of 2 greater or equal to v.
3334	static inline uint32_t VmaNextPow2(uint32_t v)
3335	{
3336	v--;
3337	v \|= v >> `1`;
3338	v \|= v >> `2`;
3339	v \|= v >> `4`;
3340	v \|= v >> `8`;
3341	v \|= v >> `16`;
3342	v++;
3343	return v;
3344	}
3345
3346	static inline uint64_t VmaNextPow2(uint64_t v)
3347	{
3348	v--;
3349	v \|= v >> `1`;
3350	v \|= v >> `2`;
3351	v \|= v >> `4`;
3352	v \|= v >> `8`;
3353	v \|= v >> `16`;
3354	v \|= v >> `32`;
3355	v++;
3356	return v;
3357	}
3358
3359	// Returns largest power of 2 less or equal to v.
3360	static inline uint32_t VmaPrevPow2(uint32_t v)
3361	{
3362	v \|= v >> `1`;
3363	v \|= v >> `2`;
3364	v \|= v >> `4`;
3365	v \|= v >> `8`;
3366	v \|= v >> `16`;
3367	v = v ^ (v >> `1`);
3368	return v;
3369	}
3370
3371	static inline uint64_t VmaPrevPow2(uint64_t v)
3372	{
3373	v \|= v >> `1`;
3374	v \|= v >> `2`;
3375	v \|= v >> `4`;
3376	v \|= v >> `8`;
3377	v \|= v >> `16`;
3378	v \|= v >> `32`;
3379	v = v ^ (v >> `1`);
3380	return v;
3381	}
3382
3383	static inline bool VmaStrIsEmpty(const char* pStr)
3384	{
3385	return pStr == VMA_NULL \|\| *pStr == `'\0'`;
3386	}
3387
3388	/*
3389	Returns true if two memory blocks occupy overlapping pages.
3390	ResourceA must be in less memory offset than ResourceB.
3391
3392	Algorithm is based on "Vulkan 1.0.39 - A Specification (with all registered Vulkan extensions)"
3393	chapter 11.6 "Resource Memory Association", paragraph "Buffer-Image Granularity".
3394	*/
3395	static inline bool VmaBlocksOnSamePage(
3396	VkDeviceSize resourceAOffset,
3397	VkDeviceSize resourceASize,
3398	VkDeviceSize resourceBOffset,
3399	VkDeviceSize pageSize)
3400	{
3401	VMA_ASSERT(resourceAOffset + resourceASize <= resourceBOffset && resourceASize > `0` && pageSize > `0`);
3402	VkDeviceSize resourceAEnd = resourceAOffset + resourceASize - `1`;
3403	VkDeviceSize resourceAEndPage = resourceAEnd & ~(pageSize - `1`);
3404	VkDeviceSize resourceBStart = resourceBOffset;
3405	VkDeviceSize resourceBStartPage = resourceBStart & ~(pageSize - `1`);
3406	return resourceAEndPage == resourceBStartPage;
3407	}
3408
3409	/*
3410	Returns true if given suballocation types could conflict and must respect
3411	VkPhysicalDeviceLimits::bufferImageGranularity. They conflict if one is buffer
3412	or linear image and another one is optimal image. If type is unknown, behave
3413	conservatively.
3414	*/
3415	static inline bool VmaIsBufferImageGranularityConflict(
3416	VmaSuballocationType suballocType1,
3417	VmaSuballocationType suballocType2)
3418	{
3419	if (suballocType1 > suballocType2)
3420	{
3421	VMA_SWAP(suballocType1, suballocType2);
3422	}
3423
3424	switch (suballocType1)
3425	{
3426	case VMA_SUBALLOCATION_TYPE_FREE:
3427	return false;
3428	case VMA_SUBALLOCATION_TYPE_UNKNOWN:
3429	return true;
3430	case VMA_SUBALLOCATION_TYPE_BUFFER:
3431	return
3432	suballocType2 == VMA_SUBALLOCATION_TYPE_IMAGE_UNKNOWN \|\|
3433	suballocType2 == VMA_SUBALLOCATION_TYPE_IMAGE_OPTIMAL;
3434	case VMA_SUBALLOCATION_TYPE_IMAGE_UNKNOWN:
3435	return
3436	suballocType2 == VMA_SUBALLOCATION_TYPE_IMAGE_UNKNOWN \|\|
3437	suballocType2 == VMA_SUBALLOCATION_TYPE_IMAGE_LINEAR \|\|
3438	suballocType2 == VMA_SUBALLOCATION_TYPE_IMAGE_OPTIMAL;
3439	case VMA_SUBALLOCATION_TYPE_IMAGE_LINEAR:
3440	return
3441	suballocType2 == VMA_SUBALLOCATION_TYPE_IMAGE_OPTIMAL;
3442	case VMA_SUBALLOCATION_TYPE_IMAGE_OPTIMAL:
3443	return false;
3444	default:
3445	VMA_ASSERT(`0`);
3446	return true;
3447	}
3448	}
3449
3450	static void VmaWriteMagicValue(void* pData, VkDeviceSize offset)
3451	{
3452	#if VMA_DEBUG_MARGIN > 0 && VMA_DEBUG_DETECT_CORRUPTION
3453	uint32_t* pDst = (uint32_t)((char**)pData + offset);
3454	const size_t numberCount = VMA_DEBUG_MARGIN / sizeof(uint32_t);
3455	for (size_t i = `0`; i < numberCount; ++i, ++pDst)
3456	{
3457	*pDst = VMA_CORRUPTION_DETECTION_MAGIC_VALUE;
3458	}
3459	#else
3460	// no-op
3461	#endif
3462	}
3463
3464	static bool VmaValidateMagicValue(const void* pData, VkDeviceSize offset)
3465	{
3466	#if VMA_DEBUG_MARGIN > 0 && VMA_DEBUG_DETECT_CORRUPTION
3467	const uint32_t* pSrc = (const uint32_t)((const* char*)pData + offset);
3468	const size_t numberCount = VMA_DEBUG_MARGIN / sizeof(uint32_t);
3469	for (size_t i = `0`; i < numberCount; ++i, ++pSrc)
3470	{
3471	if (*pSrc != VMA_CORRUPTION_DETECTION_MAGIC_VALUE)
3472	{
3473	return false;
3474	}
3475	}
3476	#endif
3477	return true;
3478	}
3479
3480	/*
3481	Fills structure with parameters of an example buffer to be used for transfers
3482	during GPU memory defragmentation.
3483	*/
3484	static void VmaFillGpuDefragmentationBufferCreateInfo(VkBufferCreateInfo& outBufCreateInfo)
3485	{
3486	memset(&outBufCreateInfo, `0`, sizeof(outBufCreateInfo));
3487	outBufCreateInfo.sType = VK_STRUCTURE_TYPE_BUFFER_CREATE_INFO;
3488	outBufCreateInfo.usage = VK_BUFFER_USAGE_TRANSFER_SRC_BIT \| VK_BUFFER_USAGE_TRANSFER_DST_BIT;
3489	outBufCreateInfo.size = (VkDeviceSize)VMA_DEFAULT_LARGE_HEAP_BLOCK_SIZE; // Example size.
3490	}
3491
3492
3493	/*
3494	Performs binary search and returns iterator to first element that is greater or
3495	equal to (key), according to comparison (cmp).
3496
3497	Cmp should return true if first argument is less than second argument.
3498
3499	Returned value is the found element, if present in the collection or place where
3500	new element with value (key) should be inserted.
3501	*/
3502	template <typename CmpLess, typename IterT, typename KeyT>
3503	static IterT VmaBinaryFindFirstNotLess(IterT beg, IterT end, const KeyT& key, const CmpLess& cmp)
3504	{
3505	size_t down = `0`, up = (end - beg);
3506	while (down < up)
3507	{
3508	const size_t mid = down + (up - down) / `2`; // Overflow-safe midpoint calculation
3509	if (cmp(*(beg + mid), key))
3510	{
3511	down = mid + `1`;
3512	}
3513	else
3514	{
3515	up = mid;
3516	}
3517	}
3518	return beg + down;
3519	}
3520
3521	template<typename CmpLess, typename IterT, typename KeyT>
3522	IterT VmaBinaryFindSorted(const IterT& beg, const IterT& end, const KeyT& value, const CmpLess& cmp)
3523	{
3524	IterT it = VmaBinaryFindFirstNotLess<CmpLess, IterT, KeyT>(
3525	beg, end, value, cmp);
3526	if (it == end \|\|
3527	(!cmp(it, value) && !cmp(value, it)))
3528	{
3529	return it;
3530	}
3531	return end;
3532	}
3533
3534	/*
3535	Returns true if all pointers in the array are not-null and unique.
3536	Warning! O(n^2) complexity. Use only inside VMA_HEAVY_ASSERT.
3537	T must be pointer type, e.g. VmaAllocation, VmaPool.
3538	*/
3539	template<typename T>
3540	static bool VmaValidatePointerArray(uint32_t count, const T* arr)
3541	{
3542	for (uint32_t i = `0`; i < count; ++i)
3543	{
3544	const T iPtr = arr[i];
3545	if (iPtr == VMA_NULL)
3546	{
3547	return false;
3548	}
3549	for (uint32_t j = i + `1`; j < count; ++j)
3550	{
3551	if (iPtr == arr[j])
3552	{
3553	return false;
3554	}
3555	}
3556	}
3557	return true;
3558	}
3559
3560	template<typename MainT, typename NewT>
3561	static inline void VmaPnextChainPushFront(MainT* mainStruct, NewT* newStruct)
3562	{
3563	newStruct->pNext = mainStruct->pNext;
3564	mainStruct->pNext = newStruct;
3565	}
3566
3567	// This is the main algorithm that guides the selection of a memory type best for an allocation -
3568	// converts usage to required/preferred/not preferred flags.
3569	static bool FindMemoryPreferences(
3570	bool isIntegratedGPU,
3571	const VmaAllocationCreateInfo& allocCreateInfo,
3572	VkFlags bufImgUsage, // VkBufferCreateInfo::usage or VkImageCreateInfo::usage. UINT32_MAX if unknown.
3573	VkMemoryPropertyFlags& outRequiredFlags,
3574	VkMemoryPropertyFlags& outPreferredFlags,
3575	VkMemoryPropertyFlags& outNotPreferredFlags)
3576	{
3577	outRequiredFlags = allocCreateInfo.requiredFlags;
3578	outPreferredFlags = allocCreateInfo.preferredFlags;
3579	outNotPreferredFlags = `0`;
3580
3581	switch(allocCreateInfo.usage)
3582	{
3583	case VMA_MEMORY_USAGE_UNKNOWN:
3584	break;
3585	case VMA_MEMORY_USAGE_GPU_ONLY:
3586	if(!isIntegratedGPU \|\| (outPreferredFlags & VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT) == `0`)
3587	{
3588	outPreferredFlags \|= VK_MEMORY_PROPERTY_DEVICE_LOCAL_BIT;
3589	}
3590	break;
3591	case VMA_MEMORY_USAGE_CPU_ONLY:
3592	outRequiredFlags \|= VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT \| VK_MEMORY_PROPERTY_HOST_COHERENT_BIT;
3593	break;
3594	case VMA_MEMORY_USAGE_CPU_TO_GPU:
3595	outRequiredFlags \|= VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT;
3596	if(!isIntegratedGPU \|\| (outPreferredFlags & VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT) == `0`)
3597	{
3598	outPreferredFlags \|= VK_MEMORY_PROPERTY_DEVICE_LOCAL_BIT;
3599	}
3600	break;
3601	case VMA_MEMORY_USAGE_GPU_TO_CPU:
3602	outRequiredFlags \|= VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT;
3603	outPreferredFlags \|= VK_MEMORY_PROPERTY_HOST_CACHED_BIT;
3604	break;
3605	case VMA_MEMORY_USAGE_CPU_COPY:
3606	outNotPreferredFlags \|= VK_MEMORY_PROPERTY_DEVICE_LOCAL_BIT;
3607	break;
3608	case VMA_MEMORY_USAGE_GPU_LAZILY_ALLOCATED:
3609	outRequiredFlags \|= VK_MEMORY_PROPERTY_LAZILY_ALLOCATED_BIT;
3610	break;
3611	case VMA_MEMORY_USAGE_AUTO:
3612	case VMA_MEMORY_USAGE_AUTO_PREFER_DEVICE:
3613	case VMA_MEMORY_USAGE_AUTO_PREFER_HOST:
3614	{
3615	if(bufImgUsage == UINT32_MAX)
3616	{
3617	VMA_ASSERT(`0` && "VMA_MEMORY_USAGE_AUTO* values can only be used with functions like vmaCreateBuffer, vmaCreateImage so that the details of the created resource are known.");
3618	return false;
3619	}
3620	// This relies on values of VK_IMAGE_USAGE_TRANSFER being the same VK_BUFFER_IMAGE_TRANSFER.
3621	const bool deviceAccess = (bufImgUsage & ~(VK_BUFFER_USAGE_TRANSFER_DST_BIT \| VK_BUFFER_USAGE_TRANSFER_SRC_BIT)) != `0`;
3622	const bool hostAccessSequentialWrite = (allocCreateInfo.flags & VMA_ALLOCATION_CREATE_HOST_ACCESS_SEQUENTIAL_WRITE_BIT) != `0`;
3623	const bool hostAccessRandom = (allocCreateInfo.flags & VMA_ALLOCATION_CREATE_HOST_ACCESS_RANDOM_BIT) != `0`;
3624	const bool hostAccessAllowTransferInstead = (allocCreateInfo.flags & VMA_ALLOCATION_CREATE_HOST_ACCESS_ALLOW_TRANSFER_INSTEAD_BIT) != `0`;
3625	const bool preferDevice = allocCreateInfo.usage == VMA_MEMORY_USAGE_AUTO_PREFER_DEVICE;
3626	const bool preferHost = allocCreateInfo.usage == VMA_MEMORY_USAGE_AUTO_PREFER_HOST;
3627
3628	// CPU random access - e.g. a buffer written to or transferred from GPU to read back on CPU.
3629	if(hostAccessRandom)
3630	{
3631	if(!isIntegratedGPU && deviceAccess && hostAccessAllowTransferInstead && !preferHost)
3632	{
3633	// Nice if it will end up in HOST_VISIBLE, but more importantly prefer DEVICE_LOCAL.
3634	// Omitting HOST_VISIBLE here is intentional.
3635	// In case there is DEVICE_LOCAL \| HOST_VISIBLE \| HOST_CACHED, it will pick that one.
3636	// Otherwise, this will give same weight to DEVICE_LOCAL as HOST_VISIBLE \| HOST_CACHED and select the former if occurs first on the list.
3637	outPreferredFlags \|= VK_MEMORY_PROPERTY_DEVICE_LOCAL_BIT \| VK_MEMORY_PROPERTY_HOST_CACHED_BIT;
3638	}
3639	else
3640	{
3641	// Always CPU memory, cached.
3642	outRequiredFlags \|= VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT \| VK_MEMORY_PROPERTY_HOST_CACHED_BIT;
3643	}
3644	}
3645	// CPU sequential write - may be CPU or host-visible GPU memory, uncached and write-combined.
3646	else if(hostAccessSequentialWrite)
3647	{
3648	// Want uncached and write-combined.
3649	outNotPreferredFlags \|= VK_MEMORY_PROPERTY_HOST_CACHED_BIT;
3650
3651	if(!isIntegratedGPU && deviceAccess && hostAccessAllowTransferInstead && !preferHost)
3652	{
3653	outPreferredFlags \|= VK_MEMORY_PROPERTY_DEVICE_LOCAL_BIT \| VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT;
3654	}
3655	else
3656	{
3657	outRequiredFlags \|= VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT;
3658	// Direct GPU access, CPU sequential write (e.g. a dynamic uniform buffer updated every frame)
3659	if(deviceAccess)
3660	{
3661	// Could go to CPU memory or GPU BAR/unified. Up to the user to decide. If no preference, choose GPU memory.
3662	if(preferHost)
3663	outNotPreferredFlags \|= VK_MEMORY_PROPERTY_DEVICE_LOCAL_BIT;
3664	else
3665	outPreferredFlags \|= VK_MEMORY_PROPERTY_DEVICE_LOCAL_BIT;
3666	}
3667	// GPU no direct access, CPU sequential write (e.g. an upload buffer to be transferred to the GPU)
3668	else
3669	{
3670	// Could go to CPU memory or GPU BAR/unified. Up to the user to decide. If no preference, choose CPU memory.
3671	if(preferDevice)
3672	outPreferredFlags \|= VK_MEMORY_PROPERTY_DEVICE_LOCAL_BIT;
3673	else
3674	outNotPreferredFlags \|= VK_MEMORY_PROPERTY_DEVICE_LOCAL_BIT;
3675	}
3676	}
3677	}
3678	// No CPU access
3679	else
3680	{
3681	// GPU access, no CPU access (e.g. a color attachment image) - prefer GPU memory
3682	if(deviceAccess)
3683	{
3684	// ...unless there is a clear preference from the user not to do so.
3685	if(preferHost)
3686	outNotPreferredFlags \|= VK_MEMORY_PROPERTY_DEVICE_LOCAL_BIT;
3687	else
3688	outPreferredFlags \|= VK_MEMORY_PROPERTY_DEVICE_LOCAL_BIT;
3689	}
3690	// No direct GPU access, no CPU access, just transfers.
3691	// It may be staging copy intended for e.g. preserving image for next frame (then better GPU memory) or
3692	// a "swap file" copy to free some GPU memory (then better CPU memory).
3693	// Up to the user to decide. If no preferece, assume the former and choose GPU memory.
3694	if(preferHost)
3695	outNotPreferredFlags \|= VK_MEMORY_PROPERTY_DEVICE_LOCAL_BIT;
3696	else
3697	outPreferredFlags \|= VK_MEMORY_PROPERTY_DEVICE_LOCAL_BIT;
3698	}
3699	break;
3700	}
3701	default:
3702	VMA_ASSERT(`0`);
3703	}
3704
3705	// Avoid DEVICE_COHERENT unless explicitly requested.
3706	if(((allocCreateInfo.requiredFlags \| allocCreateInfo.preferredFlags) &
3707	(VK_MEMORY_PROPERTY_DEVICE_COHERENT_BIT_AMD_COPY \| VK_MEMORY_PROPERTY_DEVICE_UNCACHED_BIT_AMD_COPY)) == `0`)
3708	{
3709	outNotPreferredFlags \|= VK_MEMORY_PROPERTY_DEVICE_UNCACHED_BIT_AMD_COPY;
3710	}
3711
3712	return true;
3713	}
3714
3715	////////////////////////////////////////////////////////////////////////////////
3716	// Memory allocation
3717
3718	static void* VmaMalloc(const VkAllocationCallbacks* pAllocationCallbacks, size_t size, size_t alignment)
3719	{
3720	void* result = VMA_NULL;
3721	if ((pAllocationCallbacks != VMA_NULL) &&
3722	(pAllocationCallbacks->pfnAllocation != VMA_NULL))
3723	{
3724	result = (*pAllocationCallbacks->pfnAllocation)(
3725	pAllocationCallbacks->pUserData,
3726	size,
3727	alignment,
3728	VK_SYSTEM_ALLOCATION_SCOPE_OBJECT);
3729	}
3730	else
3731	{
3732	result = VMA_SYSTEM_ALIGNED_MALLOC(size, alignment);
3733	}
3734	VMA_ASSERT(result != VMA_NULL && "CPU memory allocation failed.");
3735	return result;
3736	}
3737
3738	static void VmaFree(const VkAllocationCallbacks* pAllocationCallbacks, void* ptr)
3739	{
3740	if ((pAllocationCallbacks != VMA_NULL) &&
3741	(pAllocationCallbacks->pfnFree != VMA_NULL))
3742	{
3743	(*pAllocationCallbacks->pfnFree)(pAllocationCallbacks->pUserData, ptr);
3744	}
3745	else
3746	{
3747	VMA_SYSTEM_ALIGNED_FREE(ptr);
3748	}
3749	}
3750
3751	template<typename T>
3752	static T* VmaAllocate(const VkAllocationCallbacks* pAllocationCallbacks)
3753	{
3754	return (T)VmaMalloc(pAllocationCallbacks, sizeof*(T), VMA_ALIGN_OF(T));
3755	}
3756
3757	template<typename T>
3758	static T* VmaAllocateArray(const VkAllocationCallbacks* pAllocationCallbacks, size_t count)
3759	{
3760	return (T)VmaMalloc(pAllocationCallbacks, sizeof(T) count, VMA_ALIGN_OF(T));
3761	}
3762
3763	#define vma_new(allocator, type) new(VmaAllocate<type>(allocator))(type)
3764
3765	#define vma_new_array(allocator, type, count) new(VmaAllocateArray<type>((allocator), (count)))(type)
3766
3767	template<typename T>
3768	static void vma_delete(const VkAllocationCallbacks* pAllocationCallbacks, T* ptr)
3769	{
3770	ptr->~T();
3771	VmaFree(pAllocationCallbacks, ptr);
3772	}
3773
3774	template<typename T>
3775	static void vma_delete_array(const VkAllocationCallbacks* pAllocationCallbacks, T* ptr, size_t count)
3776	{
3777	if (ptr != VMA_NULL)
3778	{
3779	for (size_t i = count; i--; )
3780	{
3781	ptr[i].~T();
3782	}
3783	VmaFree(pAllocationCallbacks, ptr);
3784	}
3785	}
3786
3787	static char* VmaCreateStringCopy(const VkAllocationCallbacks* allocs, const char* srcStr)
3788	{
3789	if (srcStr != VMA_NULL)
3790	{
3791	const size_t len = strlen(srcStr);
3792	char* const result = vma_new_array(allocs, char, len + `1`);
3793	memcpy(result, srcStr, len + `1`);
3794	return result;
3795	}
3796	return VMA_NULL;
3797	}
3798
3799	#if VMA_STATS_STRING_ENABLED
3800	static char* VmaCreateStringCopy(const VkAllocationCallbacks* allocs, const char* srcStr, size_t strLen)
3801	{
3802	if (srcStr != VMA_NULL)
3803	{
3804	char* const result = vma_new_array(allocs, char, strLen + `1`);
3805	memcpy(result, srcStr, strLen);
3806	result[strLen] = `'\0'`;
3807	return result;
3808	}
3809	return VMA_NULL;
3810	}
3811	#endif // VMA_STATS_STRING_ENABLED
3812
3813	static void VmaFreeString(const VkAllocationCallbacks* allocs, char* str)
3814	{
3815	if (str != VMA_NULL)
3816	{
3817	const size_t len = strlen(str);
3818	vma_delete_array(allocs, str, len + `1`);
3819	}
3820	}
3821
3822	template<typename CmpLess, typename VectorT>
3823	size_t VmaVectorInsertSorted(VectorT& vector, const typename VectorT::value_type& value)
3824	{
3825	const size_t indexToInsert = VmaBinaryFindFirstNotLess(
3826	vector.data(),
3827	vector.data() + vector.size(),
3828	value,
3829	CmpLess()) - vector.data();
3830	VmaVectorInsert(vector, indexToInsert, value);
3831	return indexToInsert;
3832	}
3833
3834	template<typename CmpLess, typename VectorT>
3835	bool VmaVectorRemoveSorted(VectorT& vector, const typename VectorT::value_type& value)
3836	{
3837	CmpLess comparator;
3838	typename VectorT::iterator it = VmaBinaryFindFirstNotLess(
3839	vector.begin(),
3840	vector.end(),
3841	value,
3842	comparator);
3843	if ((it != vector.end()) && !comparator(it, value) && !comparator(value, it))
3844	{
3845	size_t indexToRemove = it - vector.begin();
3846	VmaVectorRemove(vector, indexToRemove);
3847	return true;
3848	}
3849	return false;
3850	}
3851	#endif // _VMA_FUNCTIONS
3852
3853	#ifndef _VMA_STATISTICS_FUNCTIONS
3854
3855	static void VmaClearStatistics(VmaStatistics& outStats)
3856	{
3857	outStats.blockCount = `0`;
3858	outStats.allocationCount = `0`;
3859	outStats.blockBytes = `0`;
3860	outStats.allocationBytes = `0`;
3861	}
3862
3863	static void VmaAddStatistics(VmaStatistics& inoutStats, const VmaStatistics& src)
3864	{
3865	inoutStats.blockCount += src.blockCount;
3866	inoutStats.allocationCount += src.allocationCount;
3867	inoutStats.blockBytes += src.blockBytes;
3868	inoutStats.allocationBytes += src.allocationBytes;
3869	}
3870
3871	static void VmaClearDetailedStatistics(VmaDetailedStatistics& outStats)
3872	{
3873	VmaClearStatistics(outStats.statistics);
3874	outStats.unusedRangeCount = `0`;
3875	outStats.allocationSizeMin = VK_WHOLE_SIZE;
3876	outStats.allocationSizeMax = `0`;
3877	outStats.unusedRangeSizeMin = VK_WHOLE_SIZE;
3878	outStats.unusedRangeSizeMax = `0`;
3879	}
3880
3881	static void VmaAddDetailedStatisticsAllocation(VmaDetailedStatistics& inoutStats, VkDeviceSize size)
3882	{
3883	inoutStats.statistics.allocationCount++;
3884	inoutStats.statistics.allocationBytes += size;
3885	inoutStats.allocationSizeMin = VMA_MIN(inoutStats.allocationSizeMin, size);
3886	inoutStats.allocationSizeMax = VMA_MAX(inoutStats.allocationSizeMax, size);
3887	}
3888
3889	static void VmaAddDetailedStatisticsUnusedRange(VmaDetailedStatistics& inoutStats, VkDeviceSize size)
3890	{
3891	inoutStats.unusedRangeCount++;
3892	inoutStats.unusedRangeSizeMin = VMA_MIN(inoutStats.unusedRangeSizeMin, size);
3893	inoutStats.unusedRangeSizeMax = VMA_MAX(inoutStats.unusedRangeSizeMax, size);
3894	}
3895
3896	static void VmaAddDetailedStatistics(VmaDetailedStatistics& inoutStats, const VmaDetailedStatistics& src)
3897	{
3898	VmaAddStatistics(inoutStats.statistics, src.statistics);
3899	inoutStats.unusedRangeCount += src.unusedRangeCount;
3900	inoutStats.allocationSizeMin = VMA_MIN(inoutStats.allocationSizeMin, src.allocationSizeMin);
3901	inoutStats.allocationSizeMax = VMA_MAX(inoutStats.allocationSizeMax, src.allocationSizeMax);
3902	inoutStats.unusedRangeSizeMin = VMA_MIN(inoutStats.unusedRangeSizeMin, src.unusedRangeSizeMin);
3903	inoutStats.unusedRangeSizeMax = VMA_MAX(inoutStats.unusedRangeSizeMax, src.unusedRangeSizeMax);
3904	}
3905
3906	#endif // _VMA_STATISTICS_FUNCTIONS
3907
3908	#ifndef _VMA_MUTEX_LOCK
3909	// Helper RAII class to lock a mutex in constructor and unlock it in destructor (at the end of scope).
3910	struct VmaMutexLock
3911	{
3912	VMA_CLASS_NO_COPY(VmaMutexLock)
3913	public:
3914	VmaMutexLock(VMA_MUTEX& mutex, bool useMutex = true) :
3915	m_pMutex(useMutex ? &mutex : VMA_NULL)
3916	{
3917	if (m_pMutex) { m_pMutex->Lock(); }
3918	}
3919	~VmaMutexLock() { if (m_pMutex) { m_pMutex->Unlock(); } }
3920
3921	private:
3922	VMA_MUTEX* m_pMutex;
3923	};
3924
3925	// Helper RAII class to lock a RW mutex in constructor and unlock it in destructor (at the end of scope), for reading.
3926	struct VmaMutexLockRead
3927	{
3928	VMA_CLASS_NO_COPY(VmaMutexLockRead)
3929	public:
3930	VmaMutexLockRead(VMA_RW_MUTEX& mutex, bool useMutex) :
3931	m_pMutex(useMutex ? &mutex : VMA_NULL)
3932	{
3933	if (m_pMutex) { m_pMutex->LockRead(); }
3934	}
3935	~VmaMutexLockRead() { if (m_pMutex) { m_pMutex->UnlockRead(); } }
3936
3937	private:
3938	VMA_RW_MUTEX* m_pMutex;
3939	};
3940
3941	// Helper RAII class to lock a RW mutex in constructor and unlock it in destructor (at the end of scope), for writing.
3942	struct VmaMutexLockWrite
3943	{
3944	VMA_CLASS_NO_COPY(VmaMutexLockWrite)
3945	public:
3946	VmaMutexLockWrite(VMA_RW_MUTEX& mutex, bool useMutex)
3947	: m_pMutex(useMutex ? &mutex : VMA_NULL)
3948	{
3949	if (m_pMutex) { m_pMutex->LockWrite(); }
3950	}
3951	~VmaMutexLockWrite() { if (m_pMutex) { m_pMutex->UnlockWrite(); } }
3952
3953	private:
3954	VMA_RW_MUTEX* m_pMutex;
3955	};
3956
3957	#if VMA_DEBUG_GLOBAL_MUTEX
3958	static VMA_MUTEX gDebugGlobalMutex;
3959	#define VMA_DEBUG_GLOBAL_MUTEX_LOCK VmaMutexLock debugGlobalMutexLock(gDebugGlobalMutex, true);
3960	#else
3961	#define VMA_DEBUG_GLOBAL_MUTEX_LOCK
3962	#endif
3963	#endif // _VMA_MUTEX_LOCK
3964
3965	#ifndef _VMA_ATOMIC_TRANSACTIONAL_INCREMENT
3966	// An object that increments given atomic but decrements it back in the destructor unless Commit() is called.
3967	template<typename T>
3968	struct AtomicTransactionalIncrement
3969	{
3970	public:
3971	typedef std::atomic<T> AtomicT;
3972
3973	~AtomicTransactionalIncrement()
3974	{
3975	if(m_Atomic)
3976	--(*m_Atomic);
3977	}
3978
3979	void Commit() { m_Atomic = nullptr; }
3980	T Increment(AtomicT* atomic)
3981	{
3982	m_Atomic = atomic;
3983	return m_Atomic->fetch_add(`1`);
3984	}
3985
3986	private:
3987	AtomicT* m_Atomic = nullptr;
3988	};
3989	#endif // _VMA_ATOMIC_TRANSACTIONAL_INCREMENT
3990
3991	#ifndef _VMA_STL_ALLOCATOR
3992	// STL-compatible allocator.
3993	template<typename T>
3994	struct VmaStlAllocator
3995	{
3996	const VkAllocationCallbacks* const m_pCallbacks;
3997	typedef T value_type;
3998
3999	VmaStlAllocator(const VkAllocationCallbacks* pCallbacks) : m_pCallbacks(pCallbacks) {}
4000	template<typename U>
4001	VmaStlAllocator(const VmaStlAllocator<U>& src) : m_pCallbacks(src.m_pCallbacks) {}
4002	VmaStlAllocator(const VmaStlAllocator&) = default;
4003	VmaStlAllocator& operator=(const VmaStlAllocator&) = delete;
4004
4005	T* allocate(size_t n) { return VmaAllocateArray<T>(m_pCallbacks, n); }
4006	void deallocate(T* p, size_t n) { VmaFree(m_pCallbacks, p); }
4007
4008	template<typename U>
4009	bool operator==(const VmaStlAllocator<U>& rhs) const
4010	{
4011	return m_pCallbacks == rhs.m_pCallbacks;
4012	}
4013	template<typename U>
4014	bool operator!=(const VmaStlAllocator<U>& rhs) const
4015	{
4016	return m_pCallbacks != rhs.m_pCallbacks;
4017	}
4018	};
4019	#endif // _VMA_STL_ALLOCATOR
4020
4021	#ifndef _VMA_VECTOR
4022	/ Class with interface compatible with subset of std::vector.*
4023	T must be POD because constructors and destructors are not called and memcpy is
4024	used for these objects. /*
4025	template<typename T, typename AllocatorT>
4026	class VmaVector
4027	{
4028	public:
4029	typedef T value_type;
4030	typedef T* iterator;
4031	typedef const T* const_iterator;
4032
4033	VmaVector(const AllocatorT& allocator);
4034	VmaVector(size_t count, const AllocatorT& allocator);
4035	// This version of the constructor is here for compatibility with pre-C++14 std::vector.
4036	// value is unused.
4037	VmaVector(size_t count, const T& value, const AllocatorT& allocator) : VmaVector(count, allocator) {}
4038	VmaVector(const VmaVector<T, AllocatorT>& src);
4039	VmaVector& operator=(const VmaVector& rhs);
4040	~VmaVector() { VmaFree(m_Allocator.m_pCallbacks, m_pArray); }
4041
4042	bool empty() const { return m_Count == `0`; }
4043	size_t size() const { return m_Count; }
4044	T* data() { return m_pArray; }
4045	T& front() { VMA_HEAVY_ASSERT(m_Count > `0`); return m_pArray[`0`]; }
4046	T& back() { VMA_HEAVY_ASSERT(m_Count > `0`); return m_pArray[m_Count - `1`]; }
4047	const T* data() const { return m_pArray; }
4048	const T& front() const { VMA_HEAVY_ASSERT(m_Count > `0`); return m_pArray[`0`]; }
4049	const T& back() const { VMA_HEAVY_ASSERT(m_Count > `0`); return m_pArray[m_Count - `1`]; }
4050
4051	iterator begin() { return m_pArray; }
4052	iterator end() { return m_pArray + m_Count; }
4053	const_iterator cbegin() const { return m_pArray; }
4054	const_iterator cend() const { return m_pArray + m_Count; }
4055	const_iterator begin() const { return cbegin(); }
4056	const_iterator end() const { return cend(); }
4057
4058	void pop_front() { VMA_HEAVY_ASSERT(m_Count > `0`); remove(`0`); }
4059	void pop_back() { VMA_HEAVY_ASSERT(m_Count > `0`); resize(size() - `1`); }
4060	void push_front(const T& src) { insert(`0`, src); }
4061
4062	void push_back(const T& src);
4063	void reserve(size_t newCapacity, bool freeMemory = false);
4064	void resize(size_t newCount);
4065	void clear() { resize(`0`); }
4066	void shrink_to_fit();
4067	void insert(size_t index, const T& src);
4068	void remove(size_t index);
4069
4070	T& operator[](size_t index) { VMA_HEAVY_ASSERT(index < m_Count); return m_pArray[index]; }
4071	const T& operator[](size_t index) const { VMA_HEAVY_ASSERT(index < m_Count); return m_pArray[index]; }
4072
4073	private:
4074	AllocatorT m_Allocator;
4075	T* m_pArray;
4076	size_t m_Count;
4077	size_t m_Capacity;
4078	};
4079
4080	#ifndef _VMA_VECTOR_FUNCTIONS
4081	template<typename T, typename AllocatorT>
4082	VmaVector<T, AllocatorT>::VmaVector(const AllocatorT& allocator)
4083	: m_Allocator(allocator),
4084	m_pArray(VMA_NULL),
4085	m_Count(`0`),
4086	m_Capacity(`0`) {}
4087
4088	template<typename T, typename AllocatorT>
4089	VmaVector<T, AllocatorT>::VmaVector(size_t count, const AllocatorT& allocator)
4090	: m_Allocator(allocator),
4091	m_pArray(count ? (T*)VmaAllocateArray<T>(allocator.m_pCallbacks, count) : VMA_NULL),
4092	m_Count(count),
4093	m_Capacity(count) {}
4094
4095	template<typename T, typename AllocatorT>
4096	VmaVector<T, AllocatorT>::VmaVector(const VmaVector& src)
4097	: m_Allocator(src.m_Allocator),
4098	m_pArray(src.m_Count ? (T*)VmaAllocateArray<T>(src.m_Allocator.m_pCallbacks, src.m_Count) : VMA_NULL),
4099	m_Count(src.m_Count),
4100	m_Capacity(src.m_Count)
4101	{
4102	if (m_Count != `0`)
4103	{
4104	memcpy(m_pArray, src.m_pArray, m_Count * sizeof(T));
4105	}
4106	}
4107
4108	template<typename T, typename AllocatorT>
4109	VmaVector<T, AllocatorT>& VmaVector<T, AllocatorT>::operator=(const VmaVector& rhs)
4110	{
4111	if (&rhs != this)
4112	{
4113	resize(rhs.m_Count);
4114	if (m_Count != `0`)
4115	{
4116	memcpy(m_pArray, rhs.m_pArray, m_Count * sizeof(T));
4117	}
4118	}
4119	return *this;
4120	}
4121
4122	template<typename T, typename AllocatorT>
4123	void VmaVector<T, AllocatorT>::push_back(const T& src)
4124	{
4125	const size_t newIndex = size();
4126	resize(newIndex + `1`);
4127	m_pArray[newIndex] = src;
4128	}
4129
4130	template<typename T, typename AllocatorT>
4131	void VmaVector<T, AllocatorT>::reserve(size_t newCapacity, bool freeMemory)
4132	{
4133	newCapacity = VMA_MAX(newCapacity, m_Count);
4134
4135	if ((newCapacity < m_Capacity) && !freeMemory)
4136	{
4137	newCapacity = m_Capacity;
4138	}
4139
4140	if (newCapacity != m_Capacity)
4141	{
4142	T* const newArray = newCapacity ? VmaAllocateArray<T>(m_Allocator, newCapacity) : VMA_NULL;
4143	if (m_Count != `0`)
4144	{
4145	memcpy(newArray, m_pArray, m_Count * sizeof(T));
4146	}
4147	VmaFree(m_Allocator.m_pCallbacks, m_pArray);
4148	m_Capacity = newCapacity;
4149	m_pArray = newArray;
4150	}
4151	}
4152
4153	template<typename T, typename AllocatorT>
4154	void VmaVector<T, AllocatorT>::resize(size_t newCount)
4155	{
4156	size_t newCapacity = m_Capacity;
4157	if (newCount > m_Capacity)
4158	{
4159	newCapacity = VMA_MAX(newCount, VMA_MAX(m_Capacity * `3` / `2`, (size_t)`8`));
4160	}
4161
4162	if (newCapacity != m_Capacity)
4163	{
4164	T* const newArray = newCapacity ? VmaAllocateArray<T>(m_Allocator.m_pCallbacks, newCapacity) : VMA_NULL;
4165	const size_t elementsToCopy = VMA_MIN(m_Count, newCount);
4166	if (elementsToCopy != `0`)
4167	{
4168	memcpy(newArray, m_pArray, elementsToCopy * sizeof(T));
4169	}
4170	VmaFree(m_Allocator.m_pCallbacks, m_pArray);
4171	m_Capacity = newCapacity;
4172	m_pArray = newArray;
4173	}
4174
4175	m_Count = newCount;
4176	}
4177
4178	template<typename T, typename AllocatorT>
4179	void VmaVector<T, AllocatorT>::shrink_to_fit()
4180	{
4181	if (m_Capacity > m_Count)
4182	{
4183	T* newArray = VMA_NULL;
4184	if (m_Count > `0`)
4185	{
4186	newArray = VmaAllocateArray<T>(m_Allocator.m_pCallbacks, m_Count);
4187	memcpy(newArray, m_pArray, m_Count * sizeof(T));
4188	}
4189	VmaFree(m_Allocator.m_pCallbacks, m_pArray);
4190	m_Capacity = m_Count;
4191	m_pArray = newArray;
4192	}
4193	}
4194
4195	template<typename T, typename AllocatorT>
4196	void VmaVector<T, AllocatorT>::insert(size_t index, const T& src)
4197	{
4198	VMA_HEAVY_ASSERT(index <= m_Count);
4199	const size_t oldCount = size();
4200	resize(oldCount + `1`);
4201	if (index < oldCount)
4202	{
4203	memmove(m_pArray + (index + `1`), m_pArray + index, (oldCount - index) * sizeof(T));
4204	}
4205	m_pArray[index] = src;
4206	}
4207
4208	template<typename T, typename AllocatorT>
4209	void VmaVector<T, AllocatorT>::remove(size_t index)
4210	{
4211	VMA_HEAVY_ASSERT(index < m_Count);
4212	const size_t oldCount = size();
4213	if (index < oldCount - `1`)
4214	{
4215	memmove(m_pArray + index, m_pArray + (index + `1`), (oldCount - index - `1`) * sizeof(T));
4216	}
4217	resize(oldCount - `1`);
4218	}
4219	#endif // _VMA_VECTOR_FUNCTIONS
4220
4221	template<typename T, typename allocatorT>
4222	static void VmaVectorInsert(VmaVector<T, allocatorT>& vec, size_t index, const T& item)
4223	{
4224	vec.insert(index, item);
4225	}
4226
4227	template<typename T, typename allocatorT>
4228	static void VmaVectorRemove(VmaVector<T, allocatorT>& vec, size_t index)
4229	{
4230	vec.remove(index);
4231	}
4232	#endif // _VMA_VECTOR
4233
4234	#ifndef _VMA_SMALL_VECTOR
4235	/*
4236	This is a vector (a variable-sized array), optimized for the case when the array is small.
4237
4238	It contains some number of elements in-place, which allows it to avoid heap allocation
4239	when the actual number of elements is below that threshold. This allows normal "small"
4240	cases to be fast without losing generality for large inputs.
4241	*/
4242	template<typename T, typename AllocatorT, size_t N>
4243	class VmaSmallVector
4244	{
4245	public:
4246	typedef T value_type;
4247	typedef T* iterator;
4248
4249	VmaSmallVector(const AllocatorT& allocator);
4250	VmaSmallVector(size_t count, const AllocatorT& allocator);
4251	template<typename SrcT, typename SrcAllocatorT, size_t SrcN>
4252	VmaSmallVector(const VmaSmallVector<SrcT, SrcAllocatorT, SrcN>&) = delete;
4253	template<typename SrcT, typename SrcAllocatorT, size_t SrcN>
4254	VmaSmallVector<T, AllocatorT, N>& operator=(const VmaSmallVector<SrcT, SrcAllocatorT, SrcN>&) = delete;
4255	~VmaSmallVector() = default;
4256
4257	bool empty() const { return m_Count == `0`; }
4258	size_t size() const { return m_Count; }
4259	T* data() { return m_Count > N ? m_DynamicArray.data() : m_StaticArray; }
4260	T& front() { VMA_HEAVY_ASSERT(m_Count > `0`); return data()[`0`]; }
4261	T& back() { VMA_HEAVY_ASSERT(m_Count > `0`); return data()[m_Count - `1`]; }
4262	const T* data() const { return m_Count > N ? m_DynamicArray.data() : m_StaticArray; }
4263	const T& front() const { VMA_HEAVY_ASSERT(m_Count > `0`); return data()[`0`]; }
4264	const T& back() const { VMA_HEAVY_ASSERT(m_Count > `0`); return data()[m_Count - `1`]; }
4265
4266	iterator begin() { return data(); }
4267	iterator end() { return data() + m_Count; }
4268
4269	void pop_front() { VMA_HEAVY_ASSERT(m_Count > `0`); remove(`0`); }
4270	void pop_back() { VMA_HEAVY_ASSERT(m_Count > `0`); resize(size() - `1`); }
4271	void push_front(const T& src) { insert(`0`, src); }
4272
4273	void push_back(const T& src);
4274	void resize(size_t newCount, bool freeMemory = false);
4275	void clear(bool freeMemory = false);
4276	void insert(size_t index, const T& src);
4277	void remove(size_t index);
4278
4279	T& operator[](size_t index) { VMA_HEAVY_ASSERT(index < m_Count); return data()[index]; }
4280	const T& operator[](size_t index) const { VMA_HEAVY_ASSERT(index < m_Count); return data()[index]; }
4281
4282	private:
4283	size_t m_Count;
4284	T m_StaticArray[N]; // Used when m_Size <= N
4285	VmaVector<T, AllocatorT> m_DynamicArray; // Used when m_Size > N
4286	};
4287
4288	#ifndef _VMA_SMALL_VECTOR_FUNCTIONS
4289	template<typename T, typename AllocatorT, size_t N>
4290	VmaSmallVector<T, AllocatorT, N>::VmaSmallVector(const AllocatorT& allocator)
4291	: m_Count(`0`),
4292	m_DynamicArray(allocator) {}
4293
4294	template<typename T, typename AllocatorT, size_t N>
4295	VmaSmallVector<T, AllocatorT, N>::VmaSmallVector(size_t count, const AllocatorT& allocator)
4296	: m_Count(count),
4297	m_DynamicArray(count > N ? count : `0`, allocator) {}
4298
4299	template<typename T, typename AllocatorT, size_t N>
4300	void VmaSmallVector<T, AllocatorT, N>::push_back(const T& src)
4301	{
4302	const size_t newIndex = size();
4303	resize(newIndex + `1`);
4304	data()[newIndex] = src;
4305	}
4306
4307	template<typename T, typename AllocatorT, size_t N>
4308	void VmaSmallVector<T, AllocatorT, N>::resize(size_t newCount, bool freeMemory)
4309	{
4310	if (newCount > N && m_Count > N)
4311	{
4312	// Any direction, staying in m_DynamicArray
4313	m_DynamicArray.resize(newCount);
4314	if (freeMemory)
4315	{
4316	m_DynamicArray.shrink_to_fit();
4317	}
4318	}
4319	else if (newCount > N && m_Count <= N)
4320	{
4321	// Growing, moving from m_StaticArray to m_DynamicArray
4322	m_DynamicArray.resize(newCount);
4323	if (m_Count > `0`)
4324	{
4325	memcpy(m_DynamicArray.data(), m_StaticArray, m_Count * sizeof(T));
4326	}
4327	}
4328	else if (newCount <= N && m_Count > N)
4329	{
4330	// Shrinking, moving from m_DynamicArray to m_StaticArray
4331	if (newCount > `0`)
4332	{
4333	memcpy(m_StaticArray, m_DynamicArray.data(), newCount * sizeof(T));
4334	}
4335	m_DynamicArray.resize(`0`);
4336	if (freeMemory)
4337	{
4338	m_DynamicArray.shrink_to_fit();
4339	}
4340	}
4341	else
4342	{
4343	// Any direction, staying in m_StaticArray - nothing to do here
4344	}
4345	m_Count = newCount;
4346	}
4347
4348	template<typename T, typename AllocatorT, size_t N>
4349	void VmaSmallVector<T, AllocatorT, N>::clear(bool freeMemory)
4350	{
4351	m_DynamicArray.clear();
4352	if (freeMemory)
4353	{
4354	m_DynamicArray.shrink_to_fit();
4355	}
4356	m_Count = `0`;
4357	}
4358
4359	template<typename T, typename AllocatorT, size_t N>
4360	void VmaSmallVector<T, AllocatorT, N>::insert(size_t index, const T& src)
4361	{
4362	VMA_HEAVY_ASSERT(index <= m_Count);
4363	const size_t oldCount = size();
4364	resize(oldCount + `1`);
4365	T* const dataPtr = data();
4366	if (index < oldCount)
4367	{
4368	// I know, this could be more optimal for case where memmove can be memcpy directly from m_StaticArray to m_DynamicArray.
4369	memmove(dataPtr + (index + `1`), dataPtr + index, (oldCount - index) * sizeof(T));
4370	}
4371	dataPtr[index] = src;
4372	}
4373
4374	template<typename T, typename AllocatorT, size_t N>
4375	void VmaSmallVector<T, AllocatorT, N>::remove(size_t index)
4376	{
4377	VMA_HEAVY_ASSERT(index < m_Count);
4378	const size_t oldCount = size();
4379	if (index < oldCount - `1`)
4380	{
4381	// I know, this could be more optimal for case where memmove can be memcpy directly from m_DynamicArray to m_StaticArray.
4382	T* const dataPtr = data();
4383	memmove(dataPtr + index, dataPtr + (index + `1`), (oldCount - index - `1`) * sizeof(T));
4384	}
4385	resize(oldCount - `1`);
4386	}
4387	#endif // _VMA_SMALL_VECTOR_FUNCTIONS
4388	#endif // _VMA_SMALL_VECTOR
4389
4390	#ifndef _VMA_POOL_ALLOCATOR
4391	/*
4392	Allocator for objects of type T using a list of arrays (pools) to speed up
4393	allocation. Number of elements that can be allocated is not bounded because
4394	allocator can create multiple blocks.
4395	*/
4396	template<typename T>
4397	class VmaPoolAllocator
4398	{
4399	VMA_CLASS_NO_COPY(VmaPoolAllocator)
4400	public:
4401	VmaPoolAllocator(const VkAllocationCallbacks* pAllocationCallbacks, uint32_t firstBlockCapacity);
4402	~VmaPoolAllocator();
4403	template<typename... Types> T* Alloc(Types&&... args);
4404	void Free(T* ptr);
4405
4406	private:
4407	union Item
4408	{
4409	uint32_t NextFreeIndex;
4410	alignas(T) char Value[sizeof(T)];
4411	};
4412	struct ItemBlock
4413	{
4414	Item* pItems;
4415	uint32_t Capacity;
4416	uint32_t FirstFreeIndex;
4417	};
4418
4419	const VkAllocationCallbacks* m_pAllocationCallbacks;
4420	const uint32_t m_FirstBlockCapacity;
4421	VmaVector<ItemBlock, VmaStlAllocator<ItemBlock>> m_ItemBlocks;
4422
4423	ItemBlock& CreateNewBlock();
4424	};
4425
4426	#ifndef _VMA_POOL_ALLOCATOR_FUNCTIONS
4427	template<typename T>
4428	VmaPoolAllocator<T>::VmaPoolAllocator(const VkAllocationCallbacks* pAllocationCallbacks, uint32_t firstBlockCapacity)
4429	: m_pAllocationCallbacks(pAllocationCallbacks),
4430	m_FirstBlockCapacity(firstBlockCapacity),
4431	m_ItemBlocks(VmaStlAllocator<ItemBlock>(pAllocationCallbacks))
4432	{
4433	VMA_ASSERT(m_FirstBlockCapacity > `1`);
4434	}
4435
4436	template<typename T>
4437	VmaPoolAllocator<T>::~VmaPoolAllocator()
4438	{
4439	for (size_t i = m_ItemBlocks.size(); i--;)
4440	vma_delete_array(m_pAllocationCallbacks, m_ItemBlocks[i].pItems, m_ItemBlocks[i].Capacity);
4441	m_ItemBlocks.clear();
4442	}
4443
4444	template<typename T>
4445	template<typename... Types> T* VmaPoolAllocator<T>::Alloc(Types&&... args)
4446	{
4447	for (size_t i = m_ItemBlocks.size(); i--; )
4448	{
4449	ItemBlock& block = m_ItemBlocks[i];
4450	// This block has some free items: Use first one.
4451	if (block.FirstFreeIndex != UINT32_MAX)
4452	{
4453	Item* const pItem = &block.pItems[block.FirstFreeIndex];
4454	block.FirstFreeIndex = pItem->NextFreeIndex;
4455	T* result = (T*)&pItem->Value;
4456	new(result)T(std::forward<Types>(args)...); // Explicit constructor call.
4457	return result;
4458	}
4459	}
4460
4461	// No block has free item: Create new one and use it.
4462	ItemBlock& newBlock = CreateNewBlock();
4463	Item* const pItem = &newBlock.pItems[`0`];
4464	newBlock.FirstFreeIndex = pItem->NextFreeIndex;
4465	T* result = (T*)&pItem->Value;
4466	new(result) T(std::forward<Types>(args)...); // Explicit constructor call.
4467	return result;
4468	}
4469
4470	template<typename T>
4471	void VmaPoolAllocator<T>::Free(T* ptr)
4472	{
4473	// Search all memory blocks to find ptr.
4474	for (size_t i = m_ItemBlocks.size(); i--; )
4475	{
4476	ItemBlock& block = m_ItemBlocks[i];
4477
4478	// Casting to union.
4479	Item* pItemPtr;
4480	memcpy(&pItemPtr, &ptr, sizeof(pItemPtr));
4481
4482	// Check if pItemPtr is in address range of this block.
4483	if ((pItemPtr >= block.pItems) && (pItemPtr < block.pItems + block.Capacity))
4484	{
4485	ptr->~T(); // Explicit destructor call.
4486	const uint32_t index = static_cast<uint32_t>(pItemPtr - block.pItems);
4487	pItemPtr->NextFreeIndex = block.FirstFreeIndex;
4488	block.FirstFreeIndex = index;
4489	return;
4490	}
4491	}
4492	VMA_ASSERT(`0` && "Pointer doesn't belong to this memory pool.");
4493	}
4494
4495	template<typename T>
4496	typename VmaPoolAllocator<T>::ItemBlock& VmaPoolAllocator<T>::CreateNewBlock()
4497	{
4498	const uint32_t newBlockCapacity = m_ItemBlocks.empty() ?
4499	m_FirstBlockCapacity : m_ItemBlocks.back().Capacity * `3` / `2`;
4500
4501	const ItemBlock newBlock =
4502	{
4503	vma_new_array(m_pAllocationCallbacks, Item, newBlockCapacity),
4504	newBlockCapacity,
4505	`0`
4506	};
4507
4508	m_ItemBlocks.push_back(newBlock);
4509
4510	// Setup singly-linked list of all free items in this block.
4511	for (uint32_t i = `0`; i < newBlockCapacity - `1`; ++i)
4512	newBlock.pItems[i].NextFreeIndex = i + `1`;
4513	newBlock.pItems[newBlockCapacity - `1`].NextFreeIndex = UINT32_MAX;
4514	return m_ItemBlocks.back();
4515	}
4516	#endif // _VMA_POOL_ALLOCATOR_FUNCTIONS
4517	#endif // _VMA_POOL_ALLOCATOR
4518
4519	#ifndef _VMA_RAW_LIST
4520	template<typename T>
4521	struct VmaListItem
4522	{
4523	VmaListItem* pPrev;
4524	VmaListItem* pNext;
4525	T Value;
4526	};
4527
4528	// Doubly linked list.
4529	template<typename T>
4530	class VmaRawList
4531	{
4532	VMA_CLASS_NO_COPY(VmaRawList)
4533	public:
4534	typedef VmaListItem<T> ItemType;
4535
4536	VmaRawList(const VkAllocationCallbacks* pAllocationCallbacks);
4537	// Intentionally not calling Clear, because that would be unnecessary
4538	// computations to return all items to m_ItemAllocator as free.
4539	~VmaRawList() = default;
4540
4541	size_t GetCount() const { return m_Count; }
4542	bool IsEmpty() const { return m_Count == `0`; }
4543
4544	ItemType* Front() { return m_pFront; }
4545	ItemType* Back() { return m_pBack; }
4546	const ItemType* Front() const { return m_pFront; }
4547	const ItemType* Back() const { return m_pBack; }
4548
4549	ItemType* PushFront();
4550	ItemType* PushBack();
4551	ItemType* PushFront(const T& value);
4552	ItemType* PushBack(const T& value);
4553	void PopFront();
4554	void PopBack();
4555
4556	// Item can be null - it means PushBack.
4557	ItemType* InsertBefore(ItemType* pItem);
4558	// Item can be null - it means PushFront.
4559	ItemType* InsertAfter(ItemType* pItem);
4560	ItemType* InsertBefore(ItemType* pItem, const T& value);
4561	ItemType* InsertAfter(ItemType* pItem, const T& value);
4562
4563	void Clear();
4564	void Remove(ItemType* pItem);
4565
4566	private:
4567	const VkAllocationCallbacks* const m_pAllocationCallbacks;
4568	VmaPoolAllocator<ItemType> m_ItemAllocator;
4569	ItemType* m_pFront;
4570	ItemType* m_pBack;
4571	size_t m_Count;
4572	};
4573
4574	#ifndef _VMA_RAW_LIST_FUNCTIONS
4575	template<typename T>
4576	VmaRawList<T>::VmaRawList(const VkAllocationCallbacks* pAllocationCallbacks)
4577	: m_pAllocationCallbacks(pAllocationCallbacks),
4578	m_ItemAllocator(pAllocationCallbacks, `128`),
4579	m_pFront(VMA_NULL),
4580	m_pBack(VMA_NULL),
4581	m_Count(`0`) {}
4582
4583	template<typename T>
4584	VmaListItem<T>* VmaRawList<T>::PushFront()
4585	{
4586	ItemType* const pNewItem = m_ItemAllocator.Alloc();
4587	pNewItem->pPrev = VMA_NULL;
4588	if (IsEmpty())
4589	{
4590	pNewItem->pNext = VMA_NULL;
4591	m_pFront = pNewItem;
4592	m_pBack = pNewItem;
4593	m_Count = `1`;
4594	}
4595	else
4596	{
4597	pNewItem->pNext = m_pFront;
4598	m_pFront->pPrev = pNewItem;
4599	m_pFront = pNewItem;
4600	++m_Count;
4601	}
4602	return pNewItem;
4603	}
4604
4605	template<typename T>
4606	VmaListItem<T>* VmaRawList<T>::PushBack()
4607	{
4608	ItemType* const pNewItem = m_ItemAllocator.Alloc();
4609	pNewItem->pNext = VMA_NULL;
4610	if(IsEmpty())
4611	{
4612	pNewItem->pPrev = VMA_NULL;
4613	m_pFront = pNewItem;
4614	m_pBack = pNewItem;
4615	m_Count = `1`;
4616	}
4617	else
4618	{
4619	pNewItem->pPrev = m_pBack;
4620	m_pBack->pNext = pNewItem;
4621	m_pBack = pNewItem;
4622	++m_Count;
4623	}
4624	return pNewItem;
4625	}
4626
4627	template<typename T>
4628	VmaListItem<T>* VmaRawList<T>::PushFront(const T& value)
4629	{
4630	ItemType* const pNewItem = PushFront();
4631	pNewItem->Value = value;
4632	return pNewItem;
4633	}
4634
4635	template<typename T>
4636	VmaListItem<T>* VmaRawList<T>::PushBack(const T& value)
4637	{
4638	ItemType* const pNewItem = PushBack();
4639	pNewItem->Value = value;
4640	return pNewItem;
4641	}
4642
4643	template<typename T>
4644	void VmaRawList<T>::PopFront()
4645	{
4646	VMA_HEAVY_ASSERT(m_Count > `0`);
4647	ItemType* const pFrontItem = m_pFront;
4648	ItemType* const pNextItem = pFrontItem->pNext;
4649	if (pNextItem != VMA_NULL)
4650	{
4651	pNextItem->pPrev = VMA_NULL;
4652	}
4653	m_pFront = pNextItem;
4654	m_ItemAllocator.Free(pFrontItem);
4655	--m_Count;
4656	}
4657
4658	template<typename T>
4659	void VmaRawList<T>::PopBack()
4660	{
4661	VMA_HEAVY_ASSERT(m_Count > `0`);
4662	ItemType* const pBackItem = m_pBack;
4663	ItemType* const pPrevItem = pBackItem->pPrev;
4664	if(pPrevItem != VMA_NULL)
4665	{
4666	pPrevItem->pNext = VMA_NULL;
4667	}
4668	m_pBack = pPrevItem;
4669	m_ItemAllocator.Free(pBackItem);
4670	--m_Count;
4671	}
4672
4673	template<typename T>
4674	void VmaRawList<T>::Clear()
4675	{
4676	if (IsEmpty() == false)
4677	{
4678	ItemType* pItem = m_pBack;
4679	while (pItem != VMA_NULL)
4680	{
4681	ItemType* const pPrevItem = pItem->pPrev;
4682	m_ItemAllocator.Free(pItem);
4683	pItem = pPrevItem;
4684	}
4685	m_pFront = VMA_NULL;
4686	m_pBack = VMA_NULL;
4687	m_Count = `0`;
4688	}
4689	}
4690
4691	template<typename T>
4692	void VmaRawList<T>::Remove(ItemType* pItem)
4693	{
4694	VMA_HEAVY_ASSERT(pItem != VMA_NULL);
4695	VMA_HEAVY_ASSERT(m_Count > `0`);
4696
4697	if(pItem->pPrev != VMA_NULL)
4698	{
4699	pItem->pPrev->pNext = pItem->pNext;
4700	}
4701	else
4702	{
4703	VMA_HEAVY_ASSERT(m_pFront == pItem);
4704	m_pFront = pItem->pNext;
4705	}
4706
4707	if(pItem->pNext != VMA_NULL)
4708	{
4709	pItem->pNext->pPrev = pItem->pPrev;
4710	}
4711	else
4712	{
4713	VMA_HEAVY_ASSERT(m_pBack == pItem);
4714	m_pBack = pItem->pPrev;
4715	}
4716
4717	m_ItemAllocator.Free(pItem);
4718	--m_Count;
4719	}
4720
4721	template<typename T>
4722	VmaListItem<T>* VmaRawList<T>::InsertBefore(ItemType* pItem)
4723	{
4724	if(pItem != VMA_NULL)
4725	{
4726	ItemType* const prevItem = pItem->pPrev;
4727	ItemType* const newItem = m_ItemAllocator.Alloc();
4728	newItem->pPrev = prevItem;
4729	newItem->pNext = pItem;
4730	pItem->pPrev = newItem;
4731	if(prevItem != VMA_NULL)
4732	{
4733	prevItem->pNext = newItem;
4734	}
4735	else
4736	{
4737	VMA_HEAVY_ASSERT(m_pFront == pItem);
4738	m_pFront = newItem;
4739	}
4740	++m_Count;
4741	return newItem;
4742	}
4743	else
4744	return PushBack();
4745	}
4746
4747	template<typename T>
4748	VmaListItem<T>* VmaRawList<T>::InsertAfter(ItemType* pItem)
4749	{
4750	if(pItem != VMA_NULL)
4751	{
4752	ItemType* const nextItem = pItem->pNext;
4753	ItemType* const newItem = m_ItemAllocator.Alloc();
4754	newItem->pNext = nextItem;
4755	newItem->pPrev = pItem;
4756	pItem->pNext = newItem;
4757	if(nextItem != VMA_NULL)
4758	{
4759	nextItem->pPrev = newItem;
4760	}
4761	else
4762	{
4763	VMA_HEAVY_ASSERT(m_pBack == pItem);
4764	m_pBack = newItem;
4765	}
4766	++m_Count;
4767	return newItem;
4768	}
4769	else
4770	return PushFront();
4771	}
4772
4773	template<typename T>
4774	VmaListItem<T>* VmaRawList<T>::InsertBefore(ItemType* pItem, const T& value)
4775	{
4776	ItemType* const newItem = InsertBefore(pItem);
4777	newItem->Value = value;
4778	return newItem;
4779	}
4780
4781	template<typename T>
4782	VmaListItem<T>* VmaRawList<T>::InsertAfter(ItemType* pItem, const T& value)
4783	{
4784	ItemType* const newItem = InsertAfter(pItem);
4785	newItem->Value = value;
4786	return newItem;
4787	}
4788	#endif // _VMA_RAW_LIST_FUNCTIONS
4789	#endif // _VMA_RAW_LIST
4790
4791	#ifndef _VMA_LIST
4792	template<typename T, typename AllocatorT>
4793	class VmaList
4794	{
4795	VMA_CLASS_NO_COPY(VmaList)
4796	public:
4797	class reverse_iterator;
4798	class const_iterator;
4799	class const_reverse_iterator;
4800
4801	class iterator
4802	{
4803	friend class const_iterator;
4804	friend class VmaList<T, AllocatorT>;
4805	public:
4806	iterator() : m_pList(VMA_NULL), m_pItem(VMA_NULL) {}
4807	iterator(const reverse_iterator& src) : m_pList(src.m_pList), m_pItem(src.m_pItem) {}
4808
4809	T& operator() const* { VMA_HEAVY_ASSERT(m_pItem != VMA_NULL); return m_pItem->Value; }
4810	T* operator->() const { VMA_HEAVY_ASSERT(m_pItem != VMA_NULL); return &m_pItem->Value; }
4811
4812	bool operator==(const iterator& rhs) const { VMA_HEAVY_ASSERT(m_pList == rhs.m_pList); return m_pItem == rhs.m_pItem; }
4813	bool operator!=(const iterator& rhs) const { VMA_HEAVY_ASSERT(m_pList == rhs.m_pList); return m_pItem != rhs.m_pItem; }
4814
4815	iterator operator++(int) { iterator result = *this; ++*this; return result; }
4816	iterator operator--(int) { iterator result = *this; --*this; return result; }
4817
4818	iterator& operator++() { VMA_HEAVY_ASSERT(m_pItem != VMA_NULL); m_pItem = m_pItem->pNext; return *this; }
4819	iterator& operator--();
4820
4821	private:
4822	VmaRawList<T>* m_pList;
4823	VmaListItem<T>* m_pItem;
4824
4825	iterator(VmaRawList<T>* pList, VmaListItem<T>* pItem) : m_pList(pList), m_pItem(pItem) {}
4826	};
4827	class reverse_iterator
4828	{
4829	friend class const_reverse_iterator;
4830	friend class VmaList<T, AllocatorT>;
4831	public:
4832	reverse_iterator() : m_pList(VMA_NULL), m_pItem(VMA_NULL) {}
4833	reverse_iterator(const iterator& src) : m_pList(src.m_pList), m_pItem(src.m_pItem) {}
4834
4835	T& operator() const* { VMA_HEAVY_ASSERT(m_pItem != VMA_NULL); return m_pItem->Value; }
4836	T* operator->() const { VMA_HEAVY_ASSERT(m_pItem != VMA_NULL); return &m_pItem->Value; }
4837
4838	bool operator==(const reverse_iterator& rhs) const { VMA_HEAVY_ASSERT(m_pList == rhs.m_pList); return m_pItem == rhs.m_pItem; }
4839	bool operator!=(const reverse_iterator& rhs) const { VMA_HEAVY_ASSERT(m_pList == rhs.m_pList); return m_pItem != rhs.m_pItem; }
4840
4841	reverse_iterator operator++(int) { reverse_iterator result = *this; ++* this; return result; }
4842	reverse_iterator operator--(int) { reverse_iterator result = *this; --* this; return result; }
4843
4844	reverse_iterator& operator++() { VMA_HEAVY_ASSERT(m_pItem != VMA_NULL); m_pItem = m_pItem->pPrev; return *this; }
4845	reverse_iterator& operator--();
4846
4847	private:
4848	VmaRawList<T>* m_pList;
4849	VmaListItem<T>* m_pItem;
4850
4851	reverse_iterator(VmaRawList<T>* pList, VmaListItem<T>* pItem) : m_pList(pList), m_pItem(pItem) {}
4852	};
4853	class const_iterator
4854	{
4855	friend class VmaList<T, AllocatorT>;
4856	public:
4857	const_iterator() : m_pList(VMA_NULL), m_pItem(VMA_NULL) {}
4858	const_iterator(const iterator& src) : m_pList(src.m_pList), m_pItem(src.m_pItem) {}
4859	const_iterator(const reverse_iterator& src) : m_pList(src.m_pList), m_pItem(src.m_pItem) {}
4860
4861	iterator drop_const() { return { const_cast<VmaRawList<T>>(m_pList), const_cast<VmaListItem<T>>(m_pItem) }; }
4862
4863	const T& operator() const* { VMA_HEAVY_ASSERT(m_pItem != VMA_NULL); return m_pItem->Value; }
4864	const T* operator->() const { VMA_HEAVY_ASSERT(m_pItem != VMA_NULL); return &m_pItem->Value; }
4865
4866	bool operator==(const const_iterator& rhs) const { VMA_HEAVY_ASSERT(m_pList == rhs.m_pList); return m_pItem == rhs.m_pItem; }
4867	bool operator!=(const const_iterator& rhs) const { VMA_HEAVY_ASSERT(m_pList == rhs.m_pList); return m_pItem != rhs.m_pItem; }
4868
4869	const_iterator operator++(int) { const_iterator result = *this; ++* this; return result; }
4870	const_iterator operator--(int) { const_iterator result = *this; --* this; return result; }
4871
4872	const_iterator& operator++() { VMA_HEAVY_ASSERT(m_pItem != VMA_NULL); m_pItem = m_pItem->pNext; return *this; }
4873	const_iterator& operator--();
4874
4875	private:
4876	const VmaRawList<T>* m_pList;
4877	const VmaListItem<T>* m_pItem;
4878
4879	const_iterator(const VmaRawList<T>* pList, const VmaListItem<T>* pItem) : m_pList(pList), m_pItem(pItem) {}
4880	};
4881	class const_reverse_iterator
4882	{
4883	friend class VmaList<T, AllocatorT>;
4884	public:
4885	const_reverse_iterator() : m_pList(VMA_NULL), m_pItem(VMA_NULL) {}
4886	const_reverse_iterator(const reverse_iterator& src) : m_pList(src.m_pList), m_pItem(src.m_pItem) {}
4887	const_reverse_iterator(const iterator& src) : m_pList(src.m_pList), m_pItem(src.m_pItem) {}
4888
4889	reverse_iterator drop_const() { return { const_cast<VmaRawList<T>>(m_pList), const_cast<VmaListItem<T>>(m_pItem) }; }
4890
4891	const T& operator() const* { VMA_HEAVY_ASSERT(m_pItem != VMA_NULL); return m_pItem->Value; }
4892	const T* operator->() const { VMA_HEAVY_ASSERT(m_pItem != VMA_NULL); return &m_pItem->Value; }
4893
4894	bool operator==(const const_reverse_iterator& rhs) const { VMA_HEAVY_ASSERT(m_pList == rhs.m_pList); return m_pItem == rhs.m_pItem; }
4895	bool operator!=(const const_reverse_iterator& rhs) const { VMA_HEAVY_ASSERT(m_pList == rhs.m_pList); return m_pItem != rhs.m_pItem; }
4896
4897	const_reverse_iterator operator++(int) { const_reverse_iterator result = *this; ++* this; return result; }
4898	const_reverse_iterator operator--(int) { const_reverse_iterator result = *this; --* this; return result; }
4899
4900	const_reverse_iterator& operator++() { VMA_HEAVY_ASSERT(m_pItem != VMA_NULL); m_pItem = m_pItem->pPrev; return *this; }
4901	const_reverse_iterator& operator--();
4902
4903	private:
4904	const VmaRawList<T>* m_pList;
4905	const VmaListItem<T>* m_pItem;
4906
4907	const_reverse_iterator(const VmaRawList<T>* pList, const VmaListItem<T>* pItem) : m_pList(pList), m_pItem(pItem) {}
4908	};
4909
4910	VmaList(const AllocatorT& allocator) : m_RawList(allocator.m_pCallbacks) {}
4911
4912	bool empty() const { return m_RawList.IsEmpty(); }
4913	size_t size() const { return m_RawList.GetCount(); }
4914
4915	iterator begin() { return iterator(&m_RawList, m_RawList.Front()); }
4916	iterator end() { return iterator(&m_RawList, VMA_NULL); }
4917
4918	const_iterator cbegin() const { return const_iterator(&m_RawList, m_RawList.Front()); }
4919	const_iterator cend() const { return const_iterator(&m_RawList, VMA_NULL); }
4920
4921	const_iterator begin() const { return cbegin(); }
4922	const_iterator end() const { return cend(); }
4923
4924	reverse_iterator rbegin() { return reverse_iterator(&m_RawList, m_RawList.Back()); }
4925	reverse_iterator rend() { return reverse_iterator(&m_RawList, VMA_NULL); }
4926
4927	const_reverse_iterator crbegin() const { return const_reverse_iterator(&m_RawList, m_RawList.Back()); }
4928	const_reverse_iterator crend() const { return const_reverse_iterator(&m_RawList, VMA_NULL); }
4929
4930	const_reverse_iterator rbegin() const { return crbegin(); }
4931	const_reverse_iterator rend() const { return crend(); }
4932
4933	void push_back(const T& value) { m_RawList.PushBack(value); }
4934	iterator insert(iterator it, const T& value) { return iterator(&m_RawList, m_RawList.InsertBefore(it.m_pItem, value)); }
4935
4936	void clear() { m_RawList.Clear(); }
4937	void erase(iterator it) { m_RawList.Remove(it.m_pItem); }
4938
4939	private:
4940	VmaRawList<T> m_RawList;
4941	};
4942
4943	#ifndef _VMA_LIST_FUNCTIONS
4944	template<typename T, typename AllocatorT>
4945	typename VmaList<T, AllocatorT>::iterator& VmaList<T, AllocatorT>::iterator::operator--()
4946	{
4947	if (m_pItem != VMA_NULL)
4948	{
4949	m_pItem = m_pItem->pPrev;
4950	}
4951	else
4952	{
4953	VMA_HEAVY_ASSERT(!m_pList->IsEmpty());
4954	m_pItem = m_pList->Back();
4955	}
4956	return *this;
4957	}
4958
4959	template<typename T, typename AllocatorT>
4960	typename VmaList<T, AllocatorT>::reverse_iterator& VmaList<T, AllocatorT>::reverse_iterator::operator--()
4961	{
4962	if (m_pItem != VMA_NULL)
4963	{
4964	m_pItem = m_pItem->pNext;
4965	}
4966	else
4967	{
4968	VMA_HEAVY_ASSERT(!m_pList->IsEmpty());
4969	m_pItem = m_pList->Front();
4970	}
4971	return *this;
4972	}
4973
4974	template<typename T, typename AllocatorT>
4975	typename VmaList<T, AllocatorT>::const_iterator& VmaList<T, AllocatorT>::const_iterator::operator--()
4976	{
4977	if (m_pItem != VMA_NULL)
4978	{
4979	m_pItem = m_pItem->pPrev;
4980	}
4981	else
4982	{
4983	VMA_HEAVY_ASSERT(!m_pList->IsEmpty());
4984	m_pItem = m_pList->Back();
4985	}
4986	return *this;
4987	}
4988
4989	template<typename T, typename AllocatorT>
4990	typename VmaList<T, AllocatorT>::const_reverse_iterator& VmaList<T, AllocatorT>::const_reverse_iterator::operator--()
4991	{
4992	if (m_pItem != VMA_NULL)
4993	{
4994	m_pItem = m_pItem->pNext;
4995	}
4996	else
4997	{
4998	VMA_HEAVY_ASSERT(!m_pList->IsEmpty());
4999	m_pItem = m_pList->Back();
5000	}
5001	return *this;
5002	}
5003	#endif // _VMA_LIST_FUNCTIONS
5004	#endif // _VMA_LIST
5005
5006	#ifndef _VMA_INTRUSIVE_LINKED_LIST
5007	/*
5008	Expected interface of ItemTypeTraits:
5009	struct MyItemTypeTraits
5010	{
5011	typedef MyItem ItemType;
5012	static ItemType GetPrev(const ItemType* item) { return item->myPrevPtr; }*
5013	static ItemType GetNext(const ItemType* item) { return item->myNextPtr; }*
5014	static ItemType& AccessPrev(ItemType* item) { return item->myPrevPtr; }*
5015	static ItemType& AccessNext(ItemType* item) { return item->myNextPtr; }*
5016	};
5017	*/
5018	template<typename ItemTypeTraits>
5019	class VmaIntrusiveLinkedList
5020	{
5021	public:
5022	typedef typename ItemTypeTraits::ItemType ItemType;
5023	static ItemType* GetPrev(const ItemType* item) { return ItemTypeTraits::GetPrev(item); }
5024	static ItemType* GetNext(const ItemType* item) { return ItemTypeTraits::GetNext(item); }
5025
5026	// Movable, not copyable.
5027	VmaIntrusiveLinkedList() = default;
5028	VmaIntrusiveLinkedList(VmaIntrusiveLinkedList && src);
5029	VmaIntrusiveLinkedList(const VmaIntrusiveLinkedList&) = delete;
5030	VmaIntrusiveLinkedList& operator=(VmaIntrusiveLinkedList&& src);
5031	VmaIntrusiveLinkedList& operator=(const VmaIntrusiveLinkedList&) = delete;
5032	~VmaIntrusiveLinkedList() { VMA_HEAVY_ASSERT(IsEmpty()); }
5033
5034	size_t GetCount() const { return m_Count; }
5035	bool IsEmpty() const { return m_Count == `0`; }
5036	ItemType* Front() { return m_Front; }
5037	ItemType* Back() { return m_Back; }
5038	const ItemType* Front() const { return m_Front; }
5039	const ItemType* Back() const { return m_Back; }
5040
5041	void PushBack(ItemType* item);
5042	void PushFront(ItemType* item);
5043	ItemType* PopBack();
5044	ItemType* PopFront();
5045
5046	// MyItem can be null - it means PushBack.
5047	void InsertBefore(ItemType* existingItem, ItemType* newItem);
5048	// MyItem can be null - it means PushFront.
5049	void InsertAfter(ItemType* existingItem, ItemType* newItem);
5050	void Remove(ItemType* item);
5051	void RemoveAll();
5052
5053	private:
5054	ItemType* m_Front = VMA_NULL;
5055	ItemType* m_Back = VMA_NULL;
5056	size_t m_Count = `0`;
5057	};
5058
5059	#ifndef _VMA_INTRUSIVE_LINKED_LIST_FUNCTIONS
5060	template<typename ItemTypeTraits>
5061	VmaIntrusiveLinkedList<ItemTypeTraits>::VmaIntrusiveLinkedList(VmaIntrusiveLinkedList&& src)
5062	: m_Front(src.m_Front), m_Back(src.m_Back), m_Count(src.m_Count)
5063	{
5064	src.m_Front = src.m_Back = VMA_NULL;
5065	src.m_Count = `0`;
5066	}
5067
5068	template<typename ItemTypeTraits>
5069	VmaIntrusiveLinkedList<ItemTypeTraits>& VmaIntrusiveLinkedList<ItemTypeTraits>::operator=(VmaIntrusiveLinkedList&& src)
5070	{
5071	if (&src != this)
5072	{
5073	VMA_HEAVY_ASSERT(IsEmpty());
5074	m_Front = src.m_Front;
5075	m_Back = src.m_Back;
5076	m_Count = src.m_Count;
5077	src.m_Front = src.m_Back = VMA_NULL;
5078	src.m_Count = `0`;
5079	}
5080	return *this;
5081	}
5082
5083	template<typename ItemTypeTraits>
5084	void VmaIntrusiveLinkedList<ItemTypeTraits>::PushBack(ItemType* item)
5085	{
5086	VMA_HEAVY_ASSERT(ItemTypeTraits::GetPrev(item) == VMA_NULL && ItemTypeTraits::GetNext(item) == VMA_NULL);
5087	if (IsEmpty())
5088	{
5089	m_Front = item;
5090	m_Back = item;
5091	m_Count = `1`;
5092	}
5093	else
5094	{
5095	ItemTypeTraits::AccessPrev(item) = m_Back;
5096	ItemTypeTraits::AccessNext(m_Back) = item;
5097	m_Back = item;
5098	++m_Count;
5099	}
5100	}
5101
5102	template<typename ItemTypeTraits>
5103	void VmaIntrusiveLinkedList<ItemTypeTraits>::PushFront(ItemType* item)
5104	{
5105	VMA_HEAVY_ASSERT(ItemTypeTraits::GetPrev(item) == VMA_NULL && ItemTypeTraits::GetNext(item) == VMA_NULL);
5106	if (IsEmpty())
5107	{
5108	m_Front = item;
5109	m_Back = item;
5110	m_Count = `1`;
5111	}
5112	else
5113	{
5114	ItemTypeTraits::AccessNext(item) = m_Front;
5115	ItemTypeTraits::AccessPrev(m_Front) = item;
5116	m_Front = item;
5117	++m_Count;
5118	}
5119	}
5120
5121	template<typename ItemTypeTraits>
5122	typename VmaIntrusiveLinkedList<ItemTypeTraits>::ItemType* VmaIntrusiveLinkedList<ItemTypeTraits>::PopBack()
5123	{
5124	VMA_HEAVY_ASSERT(m_Count > `0`);
5125	ItemType* const backItem = m_Back;
5126	ItemType* const prevItem = ItemTypeTraits::GetPrev(backItem);
5127	if (prevItem != VMA_NULL)
5128	{
5129	ItemTypeTraits::AccessNext(prevItem) = VMA_NULL;
5130	}
5131	m_Back = prevItem;
5132	--m_Count;
5133	ItemTypeTraits::AccessPrev(backItem) = VMA_NULL;
5134	ItemTypeTraits::AccessNext(backItem) = VMA_NULL;
5135	return backItem;
5136	}
5137
5138	template<typename ItemTypeTraits>
5139	typename VmaIntrusiveLinkedList<ItemTypeTraits>::ItemType* VmaIntrusiveLinkedList<ItemTypeTraits>::PopFront()
5140	{
5141	VMA_HEAVY_ASSERT(m_Count > `0`);
5142	ItemType* const frontItem = m_Front;
5143	ItemType* const nextItem = ItemTypeTraits::GetNext(frontItem);
5144	if (nextItem != VMA_NULL)
5145	{
5146	ItemTypeTraits::AccessPrev(nextItem) = VMA_NULL;
5147	}
5148	m_Front = nextItem;
5149	--m_Count;
5150	ItemTypeTraits::AccessPrev(frontItem) = VMA_NULL;
5151	ItemTypeTraits::AccessNext(frontItem) = VMA_NULL;
5152	return frontItem;
5153	}
5154
5155	template<typename ItemTypeTraits>
5156	void VmaIntrusiveLinkedList<ItemTypeTraits>::InsertBefore(ItemType* existingItem, ItemType* newItem)
5157	{
5158	VMA_HEAVY_ASSERT(newItem != VMA_NULL && ItemTypeTraits::GetPrev(newItem) == VMA_NULL && ItemTypeTraits::GetNext(newItem) == VMA_NULL);
5159	if (existingItem != VMA_NULL)
5160	{
5161	ItemType* const prevItem = ItemTypeTraits::GetPrev(existingItem);
5162	ItemTypeTraits::AccessPrev(newItem) = prevItem;
5163	ItemTypeTraits::AccessNext(newItem) = existingItem;
5164	ItemTypeTraits::AccessPrev(existingItem) = newItem;
5165	if (prevItem != VMA_NULL)
5166	{
5167	ItemTypeTraits::AccessNext(prevItem) = newItem;
5168	}
5169	else
5170	{
5171	VMA_HEAVY_ASSERT(m_Front == existingItem);
5172	m_Front = newItem;
5173	}
5174	++m_Count;
5175	}
5176	else
5177	PushBack(newItem);
5178	}
5179
5180	template<typename ItemTypeTraits>
5181	void VmaIntrusiveLinkedList<ItemTypeTraits>::InsertAfter(ItemType* existingItem, ItemType* newItem)
5182	{
5183	VMA_HEAVY_ASSERT(newItem != VMA_NULL && ItemTypeTraits::GetPrev(newItem) == VMA_NULL && ItemTypeTraits::GetNext(newItem) == VMA_NULL);
5184	if (existingItem != VMA_NULL)
5185	{
5186	ItemType* const nextItem = ItemTypeTraits::GetNext(existingItem);
5187	ItemTypeTraits::AccessNext(newItem) = nextItem;
5188	ItemTypeTraits::AccessPrev(newItem) = existingItem;
5189	ItemTypeTraits::AccessNext(existingItem) = newItem;
5190	if (nextItem != VMA_NULL)
5191	{
5192	ItemTypeTraits::AccessPrev(nextItem) = newItem;
5193	}
5194	else
5195	{
5196	VMA_HEAVY_ASSERT(m_Back == existingItem);
5197	m_Back = newItem;
5198	}
5199	++m_Count;
5200	}
5201	else
5202	return PushFront(newItem);
5203	}
5204
5205	template<typename ItemTypeTraits>
5206	void VmaIntrusiveLinkedList<ItemTypeTraits>::Remove(ItemType* item)
5207	{
5208	VMA_HEAVY_ASSERT(item != VMA_NULL && m_Count > `0`);
5209	if (ItemTypeTraits::GetPrev(item) != VMA_NULL)
5210	{
5211	ItemTypeTraits::AccessNext(ItemTypeTraits::AccessPrev(item)) = ItemTypeTraits::GetNext(item);
5212	}
5213	else
5214	{
5215	VMA_HEAVY_ASSERT(m_Front == item);
5216	m_Front = ItemTypeTraits::GetNext(item);
5217	}
5218
5219	if (ItemTypeTraits::GetNext(item) != VMA_NULL)
5220	{
5221	ItemTypeTraits::AccessPrev(ItemTypeTraits::AccessNext(item)) = ItemTypeTraits::GetPrev(item);
5222	}
5223	else
5224	{
5225	VMA_HEAVY_ASSERT(m_Back == item);
5226	m_Back = ItemTypeTraits::GetPrev(item);
5227	}
5228	ItemTypeTraits::AccessPrev(item) = VMA_NULL;
5229	ItemTypeTraits::AccessNext(item) = VMA_NULL;
5230	--m_Count;
5231	}
5232
5233	template<typename ItemTypeTraits>
5234	void VmaIntrusiveLinkedList<ItemTypeTraits>::RemoveAll()
5235	{
5236	if (!IsEmpty())
5237	{
5238	ItemType* item = m_Back;
5239	while (item != VMA_NULL)
5240	{
5241	ItemType* const prevItem = ItemTypeTraits::AccessPrev(item);
5242	ItemTypeTraits::AccessPrev(item) = VMA_NULL;
5243	ItemTypeTraits::AccessNext(item) = VMA_NULL;
5244	item = prevItem;
5245	}
5246	m_Front = VMA_NULL;
5247	m_Back = VMA_NULL;
5248	m_Count = `0`;
5249	}
5250	}
5251	#endif // _VMA_INTRUSIVE_LINKED_LIST_FUNCTIONS
5252	#endif // _VMA_INTRUSIVE_LINKED_LIST
5253
5254	// Unused in this version.
5255	#if 0
5256
5257	#ifndef _VMA_PAIR
5258	template<typename T1, typename T2>
5259	struct VmaPair
5260	{
5261	T1 first;
5262	T2 second;
5263
5264	VmaPair() : first(), second() {}
5265	VmaPair(const T1& firstSrc, const T2& secondSrc) : first(firstSrc), second(secondSrc) {}
5266	};
5267
5268	template<typename FirstT, typename SecondT>
5269	struct VmaPairFirstLess
5270	{
5271	bool operator()(const VmaPair<FirstT, SecondT>& lhs, const VmaPair<FirstT, SecondT>& rhs) const
5272	{
5273	return lhs.first < rhs.first;
5274	}
5275	bool operator()(const VmaPair<FirstT, SecondT>& lhs, const FirstT& rhsFirst) const
5276	{
5277	return lhs.first < rhsFirst;
5278	}
5279	};
5280	#endif // _VMA_PAIR
5281
5282	#ifndef _VMA_MAP
5283	/ Class compatible with subset of interface of std::unordered_map.*
5284	KeyT, ValueT must be POD because they will be stored in VmaVector.
5285	*/
5286	template<typename KeyT, typename ValueT>
5287	class VmaMap
5288	{
5289	public:
5290	typedef VmaPair<KeyT, ValueT> PairType;
5291	typedef PairType* iterator;
5292
5293	VmaMap(const VmaStlAllocator<PairType>& allocator) : m_Vector(allocator) {}
5294
5295	iterator begin() { return m_Vector.begin(); }
5296	iterator end() { return m_Vector.end(); }
5297	size_t size() { return m_Vector.size(); }
5298
5299	void insert(const PairType& pair);
5300	iterator find(const KeyT& key);
5301	void erase(iterator it);
5302
5303	private:
5304	VmaVector< PairType, VmaStlAllocator<PairType>> m_Vector;
5305	};
5306
5307	#ifndef _VMA_MAP_FUNCTIONS
5308	template<typename KeyT, typename ValueT>
5309	void VmaMap<KeyT, ValueT>::insert(const PairType& pair)
5310	{
5311	const size_t indexToInsert = VmaBinaryFindFirstNotLess(
5312	m_Vector.data(),
5313	m_Vector.data() + m_Vector.size(),
5314	pair,
5315	VmaPairFirstLess<KeyT, ValueT>()) - m_Vector.data();
5316	VmaVectorInsert(m_Vector, indexToInsert, pair);
5317	}
5318
5319	template<typename KeyT, typename ValueT>
5320	VmaPair<KeyT, ValueT>* VmaMap<KeyT, ValueT>::find(const KeyT& key)
5321	{
5322	PairType* it = VmaBinaryFindFirstNotLess(
5323	m_Vector.data(),
5324	m_Vector.data() + m_Vector.size(),
5325	key,
5326	VmaPairFirstLess<KeyT, ValueT>());
5327	if ((it != m_Vector.end()) && (it->first == key))
5328	{
5329	return it;
5330	}
5331	else
5332	{
5333	return m_Vector.end();
5334	}
5335	}
5336
5337	template<typename KeyT, typename ValueT>
5338	void VmaMap<KeyT, ValueT>::erase(iterator it)
5339	{
5340	VmaVectorRemove(m_Vector, it - m_Vector.begin());
5341	}
5342	#endif // _VMA_MAP_FUNCTIONS
5343	#endif // _VMA_MAP
5344
5345	#endif // #if 0
5346
5347	#if !defined(_VMA_STRING_BUILDER) && VMA_STATS_STRING_ENABLED
5348	class VmaStringBuilder
5349	{
5350	public:
5351	VmaStringBuilder(const VkAllocationCallbacks* allocationCallbacks) : m_Data(VmaStlAllocator<char>(allocationCallbacks)) {}
5352	~VmaStringBuilder() = default;
5353
5354	size_t GetLength() const { return m_Data.size(); }
5355	const char* GetData() const { return m_Data.data(); }
5356	void AddNewLine() { Add(`'\n'`); }
5357	void Add(char ch) { m_Data.push_back(ch); }
5358
5359	void Add(const char* pStr);
5360	void AddNumber(uint32_t num);
5361	void AddNumber(uint64_t num);
5362	void AddPointer(const void* ptr);
5363
5364	private:
5365	VmaVector<char, VmaStlAllocator<char>> m_Data;
5366	};
5367
5368	#ifndef _VMA_STRING_BUILDER_FUNCTIONS
5369	void VmaStringBuilder::Add(const char* pStr)
5370	{
5371	const size_t strLen = strlen(pStr);
5372	if (strLen > `0`)
5373	{
5374	const size_t oldCount = m_Data.size();
5375	m_Data.resize(oldCount + strLen);
5376	memcpy(m_Data.data() + oldCount, pStr, strLen);
5377	}
5378	}
5379
5380	void VmaStringBuilder::AddNumber(uint32_t num)
5381	{
5382	char buf[`11`];
5383	buf[`10`] = `'\0'`;
5384	char* p = &buf[`10`];
5385	do
5386	{
5387	*--p = `'0'` + (num % `10`);
5388	num /= `10`;
5389	} while (num);
5390	Add(p);
5391	}
5392
5393	void VmaStringBuilder::AddNumber(uint64_t num)
5394	{
5395	char buf[`21`];
5396	buf[`20`] = `'\0'`;
5397	char* p = &buf[`20`];
5398	do
5399	{
5400	*--p = `'0'` + (num % `10`);
5401	num /= `10`;
5402	} while (num);
5403	Add(p);
5404	}
5405
5406	void VmaStringBuilder::AddPointer(const void* ptr)
5407	{
5408	char buf[`21`];
5409	VmaPtrToStr(buf, sizeof(buf), ptr);
5410	Add(buf);
5411	}
5412	#endif //_VMA_STRING_BUILDER_FUNCTIONS
5413	#endif // _VMA_STRING_BUILDER
5414
5415	#if !defined(_VMA_JSON_WRITER) && VMA_STATS_STRING_ENABLED
5416	/*
5417	Allows to conveniently build a correct JSON document to be written to the
5418	VmaStringBuilder passed to the constructor.
5419	*/
5420	class VmaJsonWriter
5421	{
5422	VMA_CLASS_NO_COPY(VmaJsonWriter)
5423	public:
5424	// sb - string builder to write the document to. Must remain alive for the whole lifetime of this object.
5425	VmaJsonWriter(const VkAllocationCallbacks* pAllocationCallbacks, VmaStringBuilder& sb);
5426	~VmaJsonWriter();
5427
5428	// Begins object by writing "{".
5429	// Inside an object, you must call pairs of WriteString and a value, e.g.:
5430	// j.BeginObject(true); j.WriteString("A"); j.WriteNumber(1); j.WriteString("B"); j.WriteNumber(2); j.EndObject();
5431	// Will write: { "A": 1, "B": 2 }
5432	void BeginObject(bool singleLine = false);
5433	// Ends object by writing "}".
5434	void EndObject();
5435
5436	// Begins array by writing "[".
5437	// Inside an array, you can write a sequence of any values.
5438	void BeginArray(bool singleLine = false);
5439	// Ends array by writing "[".
5440	void EndArray();
5441
5442	// Writes a string value inside "".
5443	// pStr can contain any ANSI characters, including '"', new line etc. - they will be properly escaped.
5444	void WriteString(const char* pStr);
5445
5446	// Begins writing a string value.
5447	// Call BeginString, ContinueString, ContinueString, ..., EndString instead of
5448	// WriteString to conveniently build the string content incrementally, made of
5449	// parts including numbers.
5450	void BeginString(const char* pStr = VMA_NULL);
5451	// Posts next part of an open string.
5452	void ContinueString(const char* pStr);
5453	// Posts next part of an open string. The number is converted to decimal characters.
5454	void ContinueString(uint32_t n);
5455	void ContinueString(uint64_t n);
5456	void ContinueString_Size(size_t n);
5457	// Posts next part of an open string. Pointer value is converted to characters
5458	// using "%p" formatting - shown as hexadecimal number, e.g.: 000000081276Ad00
5459	void ContinueString_Pointer(const void* ptr);
5460	// Ends writing a string value by writing '"'.
5461	void EndString(const char* pStr = VMA_NULL);
5462
5463	// Writes a number value.
5464	void WriteNumber(uint32_t n);
5465	void WriteNumber(uint64_t n);
5466	void WriteSize(size_t n);
5467	// Writes a boolean value - false or true.
5468	void WriteBool(bool b);
5469	// Writes a null value.
5470	void WriteNull();
5471
5472	private:
5473	enum COLLECTION_TYPE
5474	{
5475	COLLECTION_TYPE_OBJECT,
5476	COLLECTION_TYPE_ARRAY,
5477	};
5478	struct StackItem
5479	{
5480	COLLECTION_TYPE type;
5481	uint32_t valueCount;
5482	bool singleLineMode;
5483	};
5484
5485	static const char* const INDENT;
5486
5487	VmaStringBuilder& m_SB;
5488	VmaVector< StackItem, VmaStlAllocator<StackItem> > m_Stack;
5489	bool m_InsideString;
5490
5491	// Write size_t for less than 64bits
5492	void WriteSize(size_t n, std::integral_constant<bool, false>) { m_SB.AddNumber(static_cast<uint32_t>(n)); }
5493	// Write size_t for 64bits
5494	void WriteSize(size_t n, std::integral_constant<bool, true>) { m_SB.AddNumber(static_cast<uint64_t>(n)); }
5495
5496	void BeginValue(bool isString);
5497	void WriteIndent(bool oneLess = false);
5498	};
5499	const char* const VmaJsonWriter::INDENT = " ";
5500
5501	#ifndef _VMA_JSON_WRITER_FUNCTIONS
5502	VmaJsonWriter::VmaJsonWriter(const VkAllocationCallbacks* pAllocationCallbacks, VmaStringBuilder& sb)
5503	: m_SB(sb),
5504	m_Stack(VmaStlAllocator<StackItem>(pAllocationCallbacks)),
5505	m_InsideString(false) {}
5506
5507	VmaJsonWriter::~VmaJsonWriter()
5508	{
5509	VMA_ASSERT(!m_InsideString);
5510	VMA_ASSERT(m_Stack.empty());
5511	}
5512
5513	void VmaJsonWriter::BeginObject(bool singleLine)
5514	{
5515	VMA_ASSERT(!m_InsideString);
5516
5517	BeginValue(false);
5518	m_SB.Add(`'{'`);
5519
5520	StackItem item;
5521	item.type = COLLECTION_TYPE_OBJECT;
5522	item.valueCount = `0`;
5523	item.singleLineMode = singleLine;
5524	m_Stack.push_back(item);
5525	}
5526
5527	void VmaJsonWriter::EndObject()
5528	{
5529	VMA_ASSERT(!m_InsideString);
5530
5531	WriteIndent(true);
5532	m_SB.Add(`'}'`);
5533
5534	VMA_ASSERT(!m_Stack.empty() && m_Stack.back().type == COLLECTION_TYPE_OBJECT);
5535	m_Stack.pop_back();
5536	}
5537
5538	void VmaJsonWriter::BeginArray(bool singleLine)
5539	{
5540	VMA_ASSERT(!m_InsideString);
5541
5542	BeginValue(false);
5543	m_SB.Add(`'['`);
5544
5545	StackItem item;
5546	item.type = COLLECTION_TYPE_ARRAY;
5547	item.valueCount = `0`;
5548	item.singleLineMode = singleLine;
5549	m_Stack.push_back(item);
5550	}
5551
5552	void VmaJsonWriter::EndArray()
5553	{
5554	VMA_ASSERT(!m_InsideString);
5555
5556	WriteIndent(true);
5557	m_SB.Add(`']'`);
5558
5559	VMA_ASSERT(!m_Stack.empty() && m_Stack.back().type == COLLECTION_TYPE_ARRAY);
5560	m_Stack.pop_back();
5561	}
5562
5563	void VmaJsonWriter::WriteString(const char* pStr)
5564	{
5565	BeginString(pStr);
5566	EndString();
5567	}
5568
5569	void VmaJsonWriter::BeginString(const char* pStr)
5570	{
5571	VMA_ASSERT(!m_InsideString);
5572
5573	BeginValue(true);
5574	m_SB.Add(`'"'`);
5575	m_InsideString = true;
5576	if (pStr != VMA_NULL && pStr[`0`] != `'\0'`)
5577	{
5578	ContinueString(pStr);
5579	}
5580	}
5581
5582	void VmaJsonWriter::ContinueString(const char* pStr)
5583	{
5584	VMA_ASSERT(m_InsideString);
5585
5586	const size_t strLen = strlen(pStr);
5587	for (size_t i = `0`; i < strLen; ++i)
5588	{
5589	char ch = pStr[i];
5590	if (ch == `'\\'`)
5591	{
5592	m_SB.Add("\\\\");
5593	}
5594	else if (ch == `'"'`)
5595	{
5596	m_SB.Add("\\\"");
5597	}
5598	else if (ch >= `32`)
5599	{
5600	m_SB.Add(ch);
5601	}
5602	else switch (ch)
5603	{
5604	case `'\b'`:
5605	m_SB.Add("\\b");
5606	break;
5607	case `'\f'`:
5608	m_SB.Add("\\f");
5609	break;
5610	case `'\n'`:
5611	m_SB.Add("\\n");
5612	break;
5613	case `'\r'`:
5614	m_SB.Add("\\r");
5615	break;
5616	case `'\t'`:
5617	m_SB.Add("\\t");
5618	break;
5619	default:
5620	VMA_ASSERT(`0` && "Character not currently supported.");
5621	break;
5622	}
5623	}
5624	}
5625
5626	void VmaJsonWriter::ContinueString(uint32_t n)
5627	{
5628	VMA_ASSERT(m_InsideString);
5629	m_SB.AddNumber(n);
5630	}
5631
5632	void VmaJsonWriter::ContinueString(uint64_t n)
5633	{
5634	VMA_ASSERT(m_InsideString);
5635	m_SB.AddNumber(n);
5636	}
5637
5638	void VmaJsonWriter::ContinueString_Size(size_t n)
5639	{
5640	VMA_ASSERT(m_InsideString);
5641	// Fix for AppleClang incorrect type casting
5642	// TODO: Change to if constexpr when C++17 used as minimal standard
5643	WriteSize(n, std::is_same<size_t, uint64_t>{});
5644	}
5645
5646	void VmaJsonWriter::ContinueString_Pointer(const void* ptr)
5647	{
5648	VMA_ASSERT(m_InsideString);
5649	m_SB.AddPointer(ptr);
5650	}
5651
5652	void VmaJsonWriter::EndString(const char* pStr)
5653	{
5654	VMA_ASSERT(m_InsideString);
5655	if (pStr != VMA_NULL && pStr[`0`] != `'\0'`)
5656	{
5657	ContinueString(pStr);
5658	}
5659	m_SB.Add(`'"'`);
5660	m_InsideString = false;
5661	}
5662
5663	void VmaJsonWriter::WriteNumber(uint32_t n)
5664	{
5665	VMA_ASSERT(!m_InsideString);
5666	BeginValue(false);
5667	m_SB.AddNumber(n);
5668	}
5669
5670	void VmaJsonWriter::WriteNumber(uint64_t n)
5671	{
5672	VMA_ASSERT(!m_InsideString);
5673	BeginValue(false);
5674	m_SB.AddNumber(n);
5675	}
5676
5677	void VmaJsonWriter::WriteSize(size_t n)
5678	{
5679	VMA_ASSERT(!m_InsideString);
5680	BeginValue(false);
5681	// Fix for AppleClang incorrect type casting
5682	// TODO: Change to if constexpr when C++17 used as minimal standard
5683	WriteSize(n, std::is_same<size_t, uint64_t>{});
5684	}
5685
5686	void VmaJsonWriter::WriteBool(bool b)
5687	{
5688	VMA_ASSERT(!m_InsideString);
5689	BeginValue(false);
5690	m_SB.Add(b ? "true" : "false");
5691	}
5692
5693	void VmaJsonWriter::WriteNull()
5694	{
5695	VMA_ASSERT(!m_InsideString);
5696	BeginValue(false);
5697	m_SB.Add("null");
5698	}
5699
5700	void VmaJsonWriter::BeginValue(bool isString)
5701	{
5702	if (!m_Stack.empty())
5703	{
5704	StackItem& currItem = m_Stack.back();
5705	if (currItem.type == COLLECTION_TYPE_OBJECT &&
5706	currItem.valueCount % `2` == `0`)
5707	{
5708	VMA_ASSERT(isString);
5709	}
5710
5711	if (currItem.type == COLLECTION_TYPE_OBJECT &&
5712	currItem.valueCount % `2` != `0`)
5713	{
5714	m_SB.Add(": ");
5715	}
5716	else if (currItem.valueCount > `0`)
5717	{
5718	m_SB.Add(", ");
5719	WriteIndent();
5720	}
5721	else
5722	{
5723	WriteIndent();
5724	}
5725	++currItem.valueCount;
5726	}
5727	}
5728
5729	void VmaJsonWriter::WriteIndent(bool oneLess)
5730	{
5731	if (!m_Stack.empty() && !m_Stack.back().singleLineMode)
5732	{
5733	m_SB.AddNewLine();
5734
5735	size_t count = m_Stack.size();
5736	if (count > `0` && oneLess)
5737	{
5738	--count;
5739	}
5740	for (size_t i = `0`; i < count; ++i)
5741	{
5742	m_SB.Add(INDENT);
5743	}
5744	}
5745	}
5746	#endif // _VMA_JSON_WRITER_FUNCTIONS
5747
5748	static void VmaPrintDetailedStatistics(VmaJsonWriter& json, const VmaDetailedStatistics& stat)
5749	{
5750	json.BeginObject();
5751
5752	json.WriteString("BlockCount");
5753	json.WriteNumber(stat.statistics.blockCount);
5754	json.WriteString("BlockBytes");
5755	json.WriteNumber(stat.statistics.blockBytes);
5756	json.WriteString("AllocationCount");
5757	json.WriteNumber(stat.statistics.allocationCount);
5758	json.WriteString("AllocationBytes");
5759	json.WriteNumber(stat.statistics.allocationBytes);
5760	json.WriteString("UnusedRangeCount");
5761	json.WriteNumber(stat.unusedRangeCount);
5762
5763	if (stat.statistics.allocationCount > `1`)
5764	{
5765	json.WriteString("AllocationSizeMin");
5766	json.WriteNumber(stat.allocationSizeMin);
5767	json.WriteString("AllocationSizeMax");
5768	json.WriteNumber(stat.allocationSizeMax);
5769	}
5770	if (stat.unusedRangeCount > `1`)
5771	{
5772	json.WriteString("UnusedRangeSizeMin");
5773	json.WriteNumber(stat.unusedRangeSizeMin);
5774	json.WriteString("UnusedRangeSizeMax");
5775	json.WriteNumber(stat.unusedRangeSizeMax);
5776	}
5777	json.EndObject();
5778	}
5779	#endif // _VMA_JSON_WRITER
5780
5781	#ifndef _VMA_MAPPING_HYSTERESIS
5782
5783	class VmaMappingHysteresis
5784	{
5785	VMA_CLASS_NO_COPY(VmaMappingHysteresis)
5786	public:
5787	VmaMappingHysteresis() = default;
5788
5789	uint32_t GetExtraMapping() const { return m_ExtraMapping; }
5790
5791	// Call when Map was called.
5792	// Returns true if switched to extra +1 mapping reference count.
5793	bool PostMap()
5794	{
5795	#if VMA_MAPPING_HYSTERESIS_ENABLED
5796	if(m_ExtraMapping == `0`)
5797	{
5798	++m_MajorCounter;
5799	if(m_MajorCounter >= COUNTER_MIN_EXTRA_MAPPING)
5800	{
5801	m_ExtraMapping = `1`;
5802	m_MajorCounter = `0`;
5803	m_MinorCounter = `0`;
5804	return true;
5805	}
5806	}
5807	else // m_ExtraMapping == 1
5808	PostMinorCounter();
5809	#endif // #if VMA_MAPPING_HYSTERESIS_ENABLED
5810	return false;
5811	}
5812
5813	// Call when Unmap was called.
5814	void PostUnmap()
5815	{
5816	#if VMA_MAPPING_HYSTERESIS_ENABLED
5817	if(m_ExtraMapping == `0`)
5818	++m_MajorCounter;
5819	else // m_ExtraMapping == 1
5820	PostMinorCounter();
5821	#endif // #if VMA_MAPPING_HYSTERESIS_ENABLED
5822	}
5823
5824	// Call when allocation was made from the memory block.
5825	void PostAlloc()
5826	{
5827	#if VMA_MAPPING_HYSTERESIS_ENABLED
5828	if(m_ExtraMapping == `1`)
5829	++m_MajorCounter;
5830	else // m_ExtraMapping == 0
5831	PostMinorCounter();
5832	#endif // #if VMA_MAPPING_HYSTERESIS_ENABLED
5833	}
5834
5835	// Call when allocation was freed from the memory block.
5836	// Returns true if switched to extra -1 mapping reference count.
5837	bool PostFree()
5838	{
5839	#if VMA_MAPPING_HYSTERESIS_ENABLED
5840	if(m_ExtraMapping == `1`)
5841	{
5842	++m_MajorCounter;
5843	if(m_MajorCounter >= COUNTER_MIN_EXTRA_MAPPING &&
5844	m_MajorCounter > m_MinorCounter + `1`)
5845	{
5846	m_ExtraMapping = `0`;
5847	m_MajorCounter = `0`;
5848	m_MinorCounter = `0`;
5849	return true;
5850	}
5851	}
5852	else // m_ExtraMapping == 0
5853	PostMinorCounter();
5854	#endif // #if VMA_MAPPING_HYSTERESIS_ENABLED
5855	return false;
5856	}
5857
5858	private:
5859	static const int32_t COUNTER_MIN_EXTRA_MAPPING = `7`;
5860
5861	uint32_t m_MinorCounter = `0`;
5862	uint32_t m_MajorCounter = `0`;
5863	uint32_t m_ExtraMapping = `0`; // 0 or 1.
5864
5865	void PostMinorCounter()
5866	{
5867	if(m_MinorCounter < m_MajorCounter)
5868	{
5869	++m_MinorCounter;
5870	}
5871	else if(m_MajorCounter > `0`)
5872	{
5873	--m_MajorCounter;
5874	--m_MinorCounter;
5875	}
5876	}
5877	};
5878
5879	#endif // _VMA_MAPPING_HYSTERESIS
5880
5881	#ifndef _VMA_DEVICE_MEMORY_BLOCK
5882	/*
5883	Represents a single block of device memory (`VkDeviceMemory`) with all the
5884	data about its regions (aka suballocations, #VmaAllocation), assigned and free.
5885
5886	Thread-safety:
5887	- Access to m_pMetadata must be externally synchronized.
5888	- Map, Unmap, Bind are synchronized internally.*
5889	*/
5890	class VmaDeviceMemoryBlock
5891	{
5892	VMA_CLASS_NO_COPY(VmaDeviceMemoryBlock)
5893	public:
5894	VmaBlockMetadata* m_pMetadata;
5895
5896	VmaDeviceMemoryBlock(VmaAllocator hAllocator);
5897	~VmaDeviceMemoryBlock();
5898
5899	// Always call after construction.
5900	void Init(
5901	VmaAllocator hAllocator,
5902	VmaPool hParentPool,
5903	uint32_t newMemoryTypeIndex,
5904	VkDeviceMemory newMemory,
5905	VkDeviceSize newSize,
5906	uint32_t id,
5907	uint32_t algorithm,
5908	VkDeviceSize bufferImageGranularity);
5909	// Always call before destruction.
5910	void Destroy(VmaAllocator allocator);
5911
5912	VmaPool GetParentPool() const { return m_hParentPool; }
5913	VkDeviceMemory GetDeviceMemory() const { return m_hMemory; }
5914	uint32_t GetMemoryTypeIndex() const { return m_MemoryTypeIndex; }
5915	uint32_t GetId() const { return m_Id; }
5916	void* GetMappedData() const { return m_pMappedData; }
5917	uint32_t GetMapRefCount() const { return m_MapCount; }
5918
5919	// Call when allocation/free was made from m_pMetadata.
5920	// Used for m_MappingHysteresis.
5921	void PostAlloc() { m_MappingHysteresis.PostAlloc(); }
5922	void PostFree(VmaAllocator hAllocator);
5923
5924	// Validates all data structures inside this object. If not valid, returns false.
5925	bool Validate() const;
5926	VkResult CheckCorruption(VmaAllocator hAllocator);
5927
5928	// ppData can be null.
5929	VkResult Map(VmaAllocator hAllocator, uint32_t count, void** ppData);
5930	void Unmap(VmaAllocator hAllocator, uint32_t count);
5931
5932	VkResult WriteMagicValueAfterAllocation(VmaAllocator hAllocator, VkDeviceSize allocOffset, VkDeviceSize allocSize);
5933	VkResult ValidateMagicValueAfterAllocation(VmaAllocator hAllocator, VkDeviceSize allocOffset, VkDeviceSize allocSize);
5934
5935	VkResult BindBufferMemory(
5936	const VmaAllocator hAllocator,
5937	const VmaAllocation hAllocation,
5938	VkDeviceSize allocationLocalOffset,
5939	VkBuffer hBuffer,
5940	const void* pNext);
5941	VkResult BindImageMemory(
5942	const VmaAllocator hAllocator,
5943	const VmaAllocation hAllocation,
5944	VkDeviceSize allocationLocalOffset,
5945	VkImage hImage,
5946	const void* pNext);
5947
5948	private:
5949	VmaPool m_hParentPool; // VK_NULL_HANDLE if not belongs to custom pool.
5950	uint32_t m_MemoryTypeIndex;
5951	uint32_t m_Id;
5952	VkDeviceMemory m_hMemory;
5953
5954	/*
5955	Protects access to m_hMemory so it is not used by multiple threads simultaneously, e.g. vkMapMemory, vkBindBufferMemory.
5956	Also protects m_MapCount, m_pMappedData.
5957	Allocations, deallocations, any change in m_pMetadata is protected by parent's VmaBlockVector::m_Mutex.
5958	*/
5959	VMA_MUTEX m_MapAndBindMutex;
5960	VmaMappingHysteresis m_MappingHysteresis;
5961	uint32_t m_MapCount;
5962	void* m_pMappedData;
5963	};
5964	#endif // _VMA_DEVICE_MEMORY_BLOCK
5965
5966	#ifndef _VMA_ALLOCATION_T
5967	struct VmaAllocation_T
5968	{
5969	friend struct VmaDedicatedAllocationListItemTraits;
5970
5971	enum FLAGS
5972	{
5973	FLAG_PERSISTENT_MAP = `0x01`,
5974	FLAG_MAPPING_ALLOWED = `0x02`,
5975	};
5976
5977	public:
5978	enum ALLOCATION_TYPE
5979	{
5980	ALLOCATION_TYPE_NONE,
5981	ALLOCATION_TYPE_BLOCK,
5982	ALLOCATION_TYPE_DEDICATED,
5983	};
5984
5985	// This struct is allocated using VmaPoolAllocator.
5986	VmaAllocation_T(bool mappingAllowed);
5987	~VmaAllocation_T();
5988
5989	void InitBlockAllocation(
5990	VmaDeviceMemoryBlock* block,
5991	VmaAllocHandle allocHandle,
5992	VkDeviceSize alignment,
5993	VkDeviceSize size,
5994	uint32_t memoryTypeIndex,
5995	VmaSuballocationType suballocationType,
5996	bool mapped);
5997	// pMappedData not null means allocation is created with MAPPED flag.
5998	void InitDedicatedAllocation(
5999	VmaPool hParentPool,
6000	uint32_t memoryTypeIndex,
6001	VkDeviceMemory hMemory,
6002	VmaSuballocationType suballocationType,
6003	void* pMappedData,
6004	VkDeviceSize size);
6005
6006	ALLOCATION_TYPE GetType() const { return (ALLOCATION_TYPE)m_Type; }
6007	VkDeviceSize GetAlignment() const { return m_Alignment; }
6008	VkDeviceSize GetSize() const { return m_Size; }
6009	void* GetUserData() const { return m_pUserData; }
6010	const char* GetName() const { return m_pName; }
6011	VmaSuballocationType GetSuballocationType() const { return (VmaSuballocationType)m_SuballocationType; }
6012
6013	VmaDeviceMemoryBlock* GetBlock() const { VMA_ASSERT(m_Type == ALLOCATION_TYPE_BLOCK); return m_BlockAllocation.m_Block; }
6014	uint32_t GetMemoryTypeIndex() const { return m_MemoryTypeIndex; }
6015	bool IsPersistentMap() const { return (m_Flags & FLAG_PERSISTENT_MAP) != `0`; }
6016	bool IsMappingAllowed() const { return (m_Flags & FLAG_MAPPING_ALLOWED) != `0`; }
6017
6018	void SetUserData(VmaAllocator hAllocator, void* pUserData) { m_pUserData = pUserData; }
6019	void SetName(VmaAllocator hAllocator, const char* pName);
6020	void FreeName(VmaAllocator hAllocator);
6021	uint8_t SwapBlockAllocation(VmaAllocator hAllocator, VmaAllocation allocation);
6022	VmaAllocHandle GetAllocHandle() const;
6023	VkDeviceSize GetOffset() const;
6024	VmaPool GetParentPool() const;
6025	VkDeviceMemory GetMemory() const;
6026	void* GetMappedData() const;
6027
6028	void BlockAllocMap();
6029	void BlockAllocUnmap();
6030	VkResult DedicatedAllocMap(VmaAllocator hAllocator, void** ppData);
6031	void DedicatedAllocUnmap(VmaAllocator hAllocator);
6032
6033	#if VMA_STATS_STRING_ENABLED
6034	uint32_t GetBufferImageUsage() const { return m_BufferImageUsage; }
6035
6036	void InitBufferImageUsage(uint32_t bufferImageUsage);
6037	void PrintParameters(class VmaJsonWriter& json) const;
6038	#endif
6039
6040	private:
6041	// Allocation out of VmaDeviceMemoryBlock.
6042	struct BlockAllocation
6043	{
6044	VmaDeviceMemoryBlock* m_Block;
6045	VmaAllocHandle m_AllocHandle;
6046	};
6047	// Allocation for an object that has its own private VkDeviceMemory.
6048	struct DedicatedAllocation
6049	{
6050	VmaPool m_hParentPool; // VK_NULL_HANDLE if not belongs to custom pool.
6051	VkDeviceMemory m_hMemory;
6052	void* m_pMappedData; // Not null means memory is mapped.
6053	VmaAllocation_T* m_Prev;
6054	VmaAllocation_T* m_Next;
6055	};
6056	union
6057	{
6058	// Allocation out of VmaDeviceMemoryBlock.
6059	BlockAllocation m_BlockAllocation;
6060	// Allocation for an object that has its own private VkDeviceMemory.
6061	DedicatedAllocation m_DedicatedAllocation;
6062	};
6063
6064	VkDeviceSize m_Alignment;
6065	VkDeviceSize m_Size;
6066	void* m_pUserData;
6067	char* m_pName;
6068	uint32_t m_MemoryTypeIndex;
6069	uint8_t m_Type; // ALLOCATION_TYPE
6070	uint8_t m_SuballocationType; // VmaSuballocationType
6071	// Reference counter for vmaMapMemory()/vmaUnmapMemory().
6072	uint8_t m_MapCount;
6073	uint8_t m_Flags; // enum FLAGS
6074	#if VMA_STATS_STRING_ENABLED
6075	uint32_t m_BufferImageUsage; // 0 if unknown.
6076	#endif
6077	};
6078	#endif // _VMA_ALLOCATION_T
6079
6080	#ifndef _VMA_DEDICATED_ALLOCATION_LIST_ITEM_TRAITS
6081	struct VmaDedicatedAllocationListItemTraits
6082	{
6083	typedef VmaAllocation_T ItemType;
6084
6085	static ItemType* GetPrev(const ItemType* item)
6086	{
6087	VMA_HEAVY_ASSERT(item->GetType() == VmaAllocation_T::ALLOCATION_TYPE_DEDICATED);
6088	return item->m_DedicatedAllocation.m_Prev;
6089	}
6090	static ItemType* GetNext(const ItemType* item)
6091	{
6092	VMA_HEAVY_ASSERT(item->GetType() == VmaAllocation_T::ALLOCATION_TYPE_DEDICATED);
6093	return item->m_DedicatedAllocation.m_Next;
6094	}
6095	static ItemType& AccessPrev(ItemType item)
6096	{
6097	VMA_HEAVY_ASSERT(item->GetType() == VmaAllocation_T::ALLOCATION_TYPE_DEDICATED);
6098	return item->m_DedicatedAllocation.m_Prev;
6099	}
6100	static ItemType& AccessNext(ItemType item)
6101	{
6102	VMA_HEAVY_ASSERT(item->GetType() == VmaAllocation_T::ALLOCATION_TYPE_DEDICATED);
6103	return item->m_DedicatedAllocation.m_Next;
6104	}
6105	};
6106	#endif // _VMA_DEDICATED_ALLOCATION_LIST_ITEM_TRAITS
6107
6108	#ifndef _VMA_DEDICATED_ALLOCATION_LIST
6109	/*
6110	Stores linked list of VmaAllocation_T objects.
6111	Thread-safe, synchronized internally.
6112	*/
6113	class VmaDedicatedAllocationList
6114	{
6115	public:
6116	VmaDedicatedAllocationList() {}
6117	~VmaDedicatedAllocationList();
6118
6119	void Init(bool useMutex) { m_UseMutex = useMutex; }
6120	bool Validate();
6121
6122	void AddDetailedStatistics(VmaDetailedStatistics& inoutStats);
6123	void AddStatistics(VmaStatistics& inoutStats);
6124	#if VMA_STATS_STRING_ENABLED
6125	// Writes JSON array with the list of allocations.
6126	void BuildStatsString(VmaJsonWriter& json);
6127	#endif
6128
6129	bool IsEmpty();
6130	void Register(VmaAllocation alloc);
6131	void Unregister(VmaAllocation alloc);
6132
6133	private:
6134	typedef VmaIntrusiveLinkedList<VmaDedicatedAllocationListItemTraits> DedicatedAllocationLinkedList;
6135
6136	bool m_UseMutex = true;
6137	VMA_RW_MUTEX m_Mutex;
6138	DedicatedAllocationLinkedList m_AllocationList;
6139	};
6140
6141	#ifndef _VMA_DEDICATED_ALLOCATION_LIST_FUNCTIONS
6142
6143	VmaDedicatedAllocationList::~VmaDedicatedAllocationList()
6144	{
6145	VMA_HEAVY_ASSERT(Validate());
6146
6147	if (!m_AllocationList.IsEmpty())
6148	{
6149	VMA_ASSERT(false && "Unfreed dedicated allocations found!");
6150	}
6151	}
6152
6153	bool VmaDedicatedAllocationList::Validate()
6154	{
6155	const size_t declaredCount = m_AllocationList.GetCount();
6156	size_t actualCount = `0`;
6157	VmaMutexLockRead lock(m_Mutex, m_UseMutex);
6158	for (VmaAllocation alloc = m_AllocationList.Front();
6159	alloc != VMA_NULL; alloc = m_AllocationList.GetNext(alloc))
6160	{
6161	++actualCount;
6162	}
6163	VMA_VALIDATE(actualCount == declaredCount);
6164
6165	return true;
6166	}
6167
6168	void VmaDedicatedAllocationList::AddDetailedStatistics(VmaDetailedStatistics& inoutStats)
6169	{
6170	for(auto* item = m_AllocationList.Front(); item != nullptr; item = DedicatedAllocationLinkedList::GetNext(item))
6171	{
6172	const VkDeviceSize size = item->GetSize();
6173	inoutStats.statistics.blockCount++;
6174	inoutStats.statistics.blockBytes += size;
6175	VmaAddDetailedStatisticsAllocation(inoutStats, item->GetSize());
6176	}
6177	}
6178
6179	void VmaDedicatedAllocationList::AddStatistics(VmaStatistics& inoutStats)
6180	{
6181	VmaMutexLockRead lock(m_Mutex, m_UseMutex);
6182
6183	const uint32_t allocCount = (uint32_t)m_AllocationList.GetCount();
6184	inoutStats.blockCount += allocCount;
6185	inoutStats.allocationCount += allocCount;
6186
6187	for(auto* item = m_AllocationList.Front(); item != nullptr; item = DedicatedAllocationLinkedList::GetNext(item))
6188	{
6189	const VkDeviceSize size = item->GetSize();
6190	inoutStats.blockBytes += size;
6191	inoutStats.allocationBytes += size;
6192	}
6193	}
6194
6195	#if VMA_STATS_STRING_ENABLED
6196	void VmaDedicatedAllocationList::BuildStatsString(VmaJsonWriter& json)
6197	{
6198	VmaMutexLockRead lock(m_Mutex, m_UseMutex);
6199	json.BeginArray();
6200	for (VmaAllocation alloc = m_AllocationList.Front();
6201	alloc != VMA_NULL; alloc = m_AllocationList.GetNext(alloc))
6202	{
6203	json.BeginObject(true);
6204	alloc->PrintParameters(json);
6205	json.EndObject();
6206	}
6207	json.EndArray();
6208	}
6209	#endif // VMA_STATS_STRING_ENABLED
6210
6211	bool VmaDedicatedAllocationList::IsEmpty()
6212	{
6213	VmaMutexLockRead lock(m_Mutex, m_UseMutex);
6214	return m_AllocationList.IsEmpty();
6215	}
6216
6217	void VmaDedicatedAllocationList::Register(VmaAllocation alloc)
6218	{
6219	VmaMutexLockWrite lock(m_Mutex, m_UseMutex);
6220	m_AllocationList.PushBack(alloc);
6221	}
6222
6223	void VmaDedicatedAllocationList::Unregister(VmaAllocation alloc)
6224	{
6225	VmaMutexLockWrite lock(m_Mutex, m_UseMutex);
6226	m_AllocationList.Remove(alloc);
6227	}
6228	#endif // _VMA_DEDICATED_ALLOCATION_LIST_FUNCTIONS
6229	#endif // _VMA_DEDICATED_ALLOCATION_LIST
6230
6231	#ifndef _VMA_SUBALLOCATION
6232	/*
6233	Represents a region of VmaDeviceMemoryBlock that is either assigned and returned as
6234	allocated memory block or free.
6235	*/
6236	struct VmaSuballocation
6237	{
6238	VkDeviceSize offset;
6239	VkDeviceSize size;
6240	void* userData;
6241	VmaSuballocationType type;
6242	};
6243
6244	// Comparator for offsets.
6245	struct VmaSuballocationOffsetLess
6246	{
6247	bool operator()(const VmaSuballocation& lhs, const VmaSuballocation& rhs) const
6248	{
6249	return lhs.offset < rhs.offset;
6250	}
6251	};
6252
6253	struct VmaSuballocationOffsetGreater
6254	{
6255	bool operator()(const VmaSuballocation& lhs, const VmaSuballocation& rhs) const
6256	{
6257	return lhs.offset > rhs.offset;
6258	}
6259	};
6260
6261	struct VmaSuballocationItemSizeLess
6262	{
6263	bool operator()(const VmaSuballocationList::iterator lhs,
6264	const VmaSuballocationList::iterator rhs) const
6265	{
6266	return lhs->size < rhs->size;
6267	}
6268
6269	bool operator()(const VmaSuballocationList::iterator lhs,
6270	VkDeviceSize rhsSize) const
6271	{
6272	return lhs->size < rhsSize;
6273	}
6274	};
6275	#endif // _VMA_SUBALLOCATION
6276
6277	#ifndef _VMA_ALLOCATION_REQUEST
6278	/*
6279	Parameters of planned allocation inside a VmaDeviceMemoryBlock.
6280	item points to a FREE suballocation.
6281	*/
6282	struct VmaAllocationRequest
6283	{
6284	VmaAllocHandle allocHandle;
6285	VkDeviceSize size;
6286	VmaSuballocationList::iterator item;
6287	void* customData;
6288	uint64_t algorithmData;
6289	VmaAllocationRequestType type;
6290	};
6291	#endif // _VMA_ALLOCATION_REQUEST
6292
6293	#ifndef _VMA_BLOCK_METADATA
6294	/*
6295	Data structure used for bookkeeping of allocations and unused ranges of memory
6296	in a single VkDeviceMemory block.
6297	*/
6298	class VmaBlockMetadata
6299	{
6300	public:
6301	// pAllocationCallbacks, if not null, must be owned externally - alive and unchanged for the whole lifetime of this object.
6302	VmaBlockMetadata(const VkAllocationCallbacks* pAllocationCallbacks,
6303	VkDeviceSize bufferImageGranularity, bool isVirtual);
6304	virtual ~VmaBlockMetadata() = default;
6305
6306	virtual void Init(VkDeviceSize size) { m_Size = size; }
6307	bool IsVirtual() const { return m_IsVirtual; }
6308	VkDeviceSize GetSize() const { return m_Size; }
6309
6310	// Validates all data structures inside this object. If not valid, returns false.
6311	virtual bool Validate() const = `0`;
6312	virtual size_t GetAllocationCount() const = `0`;
6313	virtual size_t GetFreeRegionsCount() const = `0`;
6314	virtual VkDeviceSize GetSumFreeSize() const = `0`;
6315	// Returns true if this block is empty - contains only single free suballocation.
6316	virtual bool IsEmpty() const = `0`;
6317	virtual void GetAllocationInfo(VmaAllocHandle allocHandle, VmaVirtualAllocationInfo& outInfo) = `0`;
6318	virtual VkDeviceSize GetAllocationOffset(VmaAllocHandle allocHandle) const = `0`;
6319	virtual void* GetAllocationUserData(VmaAllocHandle allocHandle) const = `0`;
6320
6321	virtual VmaAllocHandle GetAllocationListBegin() const = `0`;
6322	virtual VmaAllocHandle GetNextAllocation(VmaAllocHandle prevAlloc) const = `0`;
6323	virtual VkDeviceSize GetNextFreeRegionSize(VmaAllocHandle alloc) const = `0`;
6324
6325	// Shouldn't modify blockCount.
6326	virtual void AddDetailedStatistics(VmaDetailedStatistics& inoutStats) const = `0`;
6327	virtual void AddStatistics(VmaStatistics& inoutStats) const = `0`;
6328
6329	#if VMA_STATS_STRING_ENABLED
6330	virtual void PrintDetailedMap(class VmaJsonWriter& json) const = `0`;
6331	#endif
6332
6333	// Tries to find a place for suballocation with given parameters inside this block.
6334	// If succeeded, fills pAllocationRequest and returns true.
6335	// If failed, returns false.
6336	virtual bool CreateAllocationRequest(
6337	VkDeviceSize allocSize,
6338	VkDeviceSize allocAlignment,
6339	bool upperAddress,
6340	VmaSuballocationType allocType,
6341	// Always one of VMA_ALLOCATION_CREATE_STRATEGY_ or VMA_ALLOCATION_INTERNAL_STRATEGY_* flags.*
6342	uint32_t strategy,
6343	VmaAllocationRequest* pAllocationRequest) = `0`;
6344
6345	virtual VkResult CheckCorruption(const void* pBlockData) = `0`;
6346
6347	// Makes actual allocation based on request. Request must already be checked and valid.
6348	virtual void Alloc(
6349	const VmaAllocationRequest& request,
6350	VmaSuballocationType type,
6351	void* userData) = `0`;
6352
6353	// Frees suballocation assigned to given memory region.
6354	virtual void Free(VmaAllocHandle allocHandle) = `0`;
6355
6356	// Frees all allocations.
6357	// Careful! Don't call it if there are VmaAllocation objects owned by userData of cleared allocations!
6358	virtual void Clear() = `0`;
6359
6360	virtual void SetAllocationUserData(VmaAllocHandle allocHandle, void* userData) = `0`;
6361	virtual void DebugLogAllAllocations() const = `0`;
6362
6363	protected:
6364	const VkAllocationCallbacks* GetAllocationCallbacks() const { return m_pAllocationCallbacks; }
6365	VkDeviceSize GetBufferImageGranularity() const { return m_BufferImageGranularity; }
6366	VkDeviceSize GetDebugMargin() const { return IsVirtual() ? `0` : VMA_DEBUG_MARGIN; }
6367
6368	void DebugLogAllocation(VkDeviceSize offset, VkDeviceSize size, void* userData) const;
6369	#if VMA_STATS_STRING_ENABLED
6370	// mapRefCount == UINT32_MAX means unspecified.
6371	void PrintDetailedMap_Begin(class VmaJsonWriter& json,
6372	VkDeviceSize unusedBytes,
6373	size_t allocationCount,
6374	size_t unusedRangeCount) const;
6375	void PrintDetailedMap_Allocation(class VmaJsonWriter& json,
6376	VkDeviceSize offset, VkDeviceSize size, void* userData) const;
6377	void PrintDetailedMap_UnusedRange(class VmaJsonWriter& json,
6378	VkDeviceSize offset,
6379	VkDeviceSize size) const;
6380	void PrintDetailedMap_End(class VmaJsonWriter& json) const;
6381	#endif
6382
6383	private:
6384	VkDeviceSize m_Size;
6385	const VkAllocationCallbacks* m_pAllocationCallbacks;
6386	const VkDeviceSize m_BufferImageGranularity;
6387	const bool m_IsVirtual;
6388	};
6389
6390	#ifndef _VMA_BLOCK_METADATA_FUNCTIONS
6391	VmaBlockMetadata::VmaBlockMetadata(const VkAllocationCallbacks* pAllocationCallbacks,
6392	VkDeviceSize bufferImageGranularity, bool isVirtual)
6393	: m_Size(`0`),
6394	m_pAllocationCallbacks(pAllocationCallbacks),
6395	m_BufferImageGranularity(bufferImageGranularity),
6396	m_IsVirtual(isVirtual) {}
6397
6398	void VmaBlockMetadata::DebugLogAllocation(VkDeviceSize offset, VkDeviceSize size, void* userData) const
6399	{
6400	if (IsVirtual())
6401	{
6402	VMA_DEBUG_LOG("UNFREED VIRTUAL ALLOCATION; Offset: %llu; Size: %llu; UserData: %p", offset, size, userData);
6403	}
6404	else
6405	{
6406	VMA_ASSERT(userData != VMA_NULL);
6407	VmaAllocation allocation = reinterpret_cast<VmaAllocation>(userData);
6408
6409	userData = allocation->GetUserData();
6410	const char* name = allocation->GetName();
6411
6412	#if VMA_STATS_STRING_ENABLED
6413	VMA_DEBUG_LOG("UNFREED ALLOCATION; Offset: %llu; Size: %llu; UserData: %p; Name: %s; Type: %s; Usage: %u",
6414	offset, size, userData, name ? name : "vma_empty",
6415	VMA_SUBALLOCATION_TYPE_NAMES[allocation->GetSuballocationType()],
6416	allocation->GetBufferImageUsage());
6417	#else
6418	VMA_DEBUG_LOG("UNFREED ALLOCATION; Offset: %llu; Size: %llu; UserData: %p; Name: %s; Type: %u",
6419	offset, size, userData, name ? name : "vma_empty",
6420	(uint32_t)allocation->GetSuballocationType());
6421	#endif // VMA_STATS_STRING_ENABLED
6422	}
6423
6424	}
6425
6426	#if VMA_STATS_STRING_ENABLED
6427	void VmaBlockMetadata::PrintDetailedMap_Begin(class VmaJsonWriter& json,
6428	VkDeviceSize unusedBytes, size_t allocationCount, size_t unusedRangeCount) const
6429	{
6430	json.WriteString("TotalBytes");
6431	json.WriteNumber(GetSize());
6432
6433	json.WriteString("UnusedBytes");
6434	json.WriteSize(unusedBytes);
6435
6436	json.WriteString("Allocations");
6437	json.WriteSize(allocationCount);
6438
6439	json.WriteString("UnusedRanges");
6440	json.WriteSize(unusedRangeCount);
6441
6442	json.WriteString("Suballocations");
6443	json.BeginArray();
6444	}
6445
6446	void VmaBlockMetadata::PrintDetailedMap_Allocation(class VmaJsonWriter& json,
6447	VkDeviceSize offset, VkDeviceSize size, void* userData) const
6448	{
6449	json.BeginObject(true);
6450
6451	json.WriteString("Offset");
6452	json.WriteNumber(offset);
6453
6454	if (IsVirtual())
6455	{
6456	json.WriteString("Size");
6457	json.WriteNumber(size);
6458	if (userData)
6459	{
6460	json.WriteString("CustomData");
6461	json.BeginString();
6462	json.ContinueString_Pointer(userData);
6463	json.EndString();
6464	}
6465	}
6466	else
6467	{
6468	((VmaAllocation)userData)->PrintParameters(json);
6469	}
6470
6471	json.EndObject();
6472	}
6473
6474	void VmaBlockMetadata::PrintDetailedMap_UnusedRange(class VmaJsonWriter& json,
6475	VkDeviceSize offset, VkDeviceSize size) const
6476	{
6477	json.BeginObject(true);
6478
6479	json.WriteString("Offset");
6480	json.WriteNumber(offset);
6481
6482	json.WriteString("Type");
6483	json.WriteString(VMA_SUBALLOCATION_TYPE_NAMES[VMA_SUBALLOCATION_TYPE_FREE]);
6484
6485	json.WriteString("Size");
6486	json.WriteNumber(size);
6487
6488	json.EndObject();
6489	}
6490
6491	void VmaBlockMetadata::PrintDetailedMap_End(class VmaJsonWriter& json) const
6492	{
6493	json.EndArray();
6494	}
6495	#endif // VMA_STATS_STRING_ENABLED
6496	#endif // _VMA_BLOCK_METADATA_FUNCTIONS
6497	#endif // _VMA_BLOCK_METADATA
6498
6499	#ifndef _VMA_BLOCK_BUFFER_IMAGE_GRANULARITY
6500	// Before deleting object of this class remember to call 'Destroy()'
6501	class VmaBlockBufferImageGranularity final
6502	{
6503	public:
6504	struct ValidationContext
6505	{
6506	const VkAllocationCallbacks* allocCallbacks;
6507	uint16_t* pageAllocs;
6508	};
6509
6510	VmaBlockBufferImageGranularity(VkDeviceSize bufferImageGranularity);
6511	~VmaBlockBufferImageGranularity();
6512
6513	bool IsEnabled() const { return m_BufferImageGranularity > MAX_LOW_BUFFER_IMAGE_GRANULARITY; }
6514
6515	void Init(const VkAllocationCallbacks* pAllocationCallbacks, VkDeviceSize size);
6516	// Before destroying object you must call free it's memory
6517	void Destroy(const VkAllocationCallbacks* pAllocationCallbacks);
6518
6519	void RoundupAllocRequest(VmaSuballocationType allocType,
6520	VkDeviceSize& inOutAllocSize,
6521	VkDeviceSize& inOutAllocAlignment) const;
6522
6523	bool CheckConflictAndAlignUp(VkDeviceSize& inOutAllocOffset,
6524	VkDeviceSize allocSize,
6525	VkDeviceSize blockOffset,
6526	VkDeviceSize blockSize,
6527	VmaSuballocationType allocType) const;
6528
6529	void AllocPages(uint8_t allocType, VkDeviceSize offset, VkDeviceSize size);
6530	void FreePages(VkDeviceSize offset, VkDeviceSize size);
6531	void Clear();
6532
6533	ValidationContext StartValidation(const VkAllocationCallbacks* pAllocationCallbacks,
6534	bool isVirutal) const;
6535	bool Validate(ValidationContext& ctx, VkDeviceSize offset, VkDeviceSize size) const;
6536	bool FinishValidation(ValidationContext& ctx) const;
6537
6538	private:
6539	static const uint16_t MAX_LOW_BUFFER_IMAGE_GRANULARITY = `256`;
6540
6541	struct RegionInfo
6542	{
6543	uint8_t allocType;
6544	uint16_t allocCount;
6545	};
6546
6547	VkDeviceSize m_BufferImageGranularity;
6548	uint32_t m_RegionCount;
6549	RegionInfo* m_RegionInfo;
6550
6551	uint32_t GetStartPage(VkDeviceSize offset) const { return OffsetToPageIndex(offset & ~(m_BufferImageGranularity - `1`)); }
6552	uint32_t GetEndPage(VkDeviceSize offset, VkDeviceSize size) const { return OffsetToPageIndex((offset + size - `1`) & ~(m_BufferImageGranularity - `1`)); }
6553
6554	uint32_t OffsetToPageIndex(VkDeviceSize offset) const;
6555	void AllocPage(RegionInfo& page, uint8_t allocType);
6556	};
6557
6558	#ifndef _VMA_BLOCK_BUFFER_IMAGE_GRANULARITY_FUNCTIONS
6559	VmaBlockBufferImageGranularity::VmaBlockBufferImageGranularity(VkDeviceSize bufferImageGranularity)
6560	: m_BufferImageGranularity(bufferImageGranularity),
6561	m_RegionCount(`0`),
6562	m_RegionInfo(VMA_NULL) {}
6563
6564	VmaBlockBufferImageGranularity::~VmaBlockBufferImageGranularity()
6565	{
6566	VMA_ASSERT(m_RegionInfo == VMA_NULL && "Free not called before destroying object!");
6567	}
6568
6569	void VmaBlockBufferImageGranularity::Init(const VkAllocationCallbacks* pAllocationCallbacks, VkDeviceSize size)
6570	{
6571	if (IsEnabled())
6572	{
6573	m_RegionCount = static_cast<uint32_t>(VmaDivideRoundingUp(size, m_BufferImageGranularity));
6574	m_RegionInfo = vma_new_array(pAllocationCallbacks, RegionInfo, m_RegionCount);
6575	memset(m_RegionInfo, `0`, m_RegionCount * sizeof(RegionInfo));
6576	}
6577	}
6578
6579	void VmaBlockBufferImageGranularity::Destroy(const VkAllocationCallbacks* pAllocationCallbacks)
6580	{
6581	if (m_RegionInfo)
6582	{
6583	vma_delete_array(pAllocationCallbacks, m_RegionInfo, m_RegionCount);
6584	m_RegionInfo = VMA_NULL;
6585	}
6586	}
6587
6588	void VmaBlockBufferImageGranularity::RoundupAllocRequest(VmaSuballocationType allocType,
6589	VkDeviceSize& inOutAllocSize,
6590	VkDeviceSize& inOutAllocAlignment) const
6591	{
6592	if (m_BufferImageGranularity > `1` &&
6593	m_BufferImageGranularity <= MAX_LOW_BUFFER_IMAGE_GRANULARITY)
6594	{
6595	if (allocType == VMA_SUBALLOCATION_TYPE_UNKNOWN \|\|
6596	allocType == VMA_SUBALLOCATION_TYPE_IMAGE_UNKNOWN \|\|
6597	allocType == VMA_SUBALLOCATION_TYPE_IMAGE_OPTIMAL)
6598	{
6599	inOutAllocAlignment = VMA_MAX(inOutAllocAlignment, m_BufferImageGranularity);
6600	inOutAllocSize = VmaAlignUp(inOutAllocSize, m_BufferImageGranularity);
6601	}
6602	}
6603	}
6604
6605	bool VmaBlockBufferImageGranularity::CheckConflictAndAlignUp(VkDeviceSize& inOutAllocOffset,
6606	VkDeviceSize allocSize,
6607	VkDeviceSize blockOffset,
6608	VkDeviceSize blockSize,
6609	VmaSuballocationType allocType) const
6610	{
6611	if (IsEnabled())
6612	{
6613	uint32_t startPage = GetStartPage(inOutAllocOffset);
6614	if (m_RegionInfo[startPage].allocCount > `0` &&
6615	VmaIsBufferImageGranularityConflict(static_cast<VmaSuballocationType>(m_RegionInfo[startPage].allocType), allocType))
6616	{
6617	inOutAllocOffset = VmaAlignUp(inOutAllocOffset, m_BufferImageGranularity);
6618	if (blockSize < allocSize + inOutAllocOffset - blockOffset)
6619	return true;
6620	++startPage;
6621	}
6622	uint32_t endPage = GetEndPage(inOutAllocOffset, allocSize);
6623	if (endPage != startPage &&
6624	m_RegionInfo[endPage].allocCount > `0` &&
6625	VmaIsBufferImageGranularityConflict(static_cast<VmaSuballocationType>(m_RegionInfo[endPage].allocType), allocType))
6626	{
6627	return true;
6628	}
6629	}
6630	return false;
6631	}
6632
6633	void VmaBlockBufferImageGranularity::AllocPages(uint8_t allocType, VkDeviceSize offset, VkDeviceSize size)
6634	{
6635	if (IsEnabled())
6636	{
6637	uint32_t startPage = GetStartPage(offset);
6638	AllocPage(m_RegionInfo[startPage], allocType);
6639
6640	uint32_t endPage = GetEndPage(offset, size);
6641	if (startPage != endPage)
6642	AllocPage(m_RegionInfo[endPage], allocType);
6643	}
6644	}
6645
6646	void VmaBlockBufferImageGranularity::FreePages(VkDeviceSize offset, VkDeviceSize size)
6647	{
6648	if (IsEnabled())
6649	{
6650	uint32_t startPage = GetStartPage(offset);
6651	--m_RegionInfo[startPage].allocCount;
6652	if (m_RegionInfo[startPage].allocCount == `0`)
6653	m_RegionInfo[startPage].allocType = VMA_SUBALLOCATION_TYPE_FREE;
6654	uint32_t endPage = GetEndPage(offset, size);
6655	if (startPage != endPage)
6656	{
6657	--m_RegionInfo[endPage].allocCount;
6658	if (m_RegionInfo[endPage].allocCount == `0`)
6659	m_RegionInfo[endPage].allocType = VMA_SUBALLOCATION_TYPE_FREE;
6660	}
6661	}
6662	}
6663
6664	void VmaBlockBufferImageGranularity::Clear()
6665	{
6666	if (m_RegionInfo)
6667	memset(m_RegionInfo, `0`, m_RegionCount * sizeof(RegionInfo));
6668	}
6669
6670	VmaBlockBufferImageGranularity::ValidationContext VmaBlockBufferImageGranularity::StartValidation(
6671	const VkAllocationCallbacks* pAllocationCallbacks, bool isVirutal) const
6672	{
6673	ValidationContext ctx{ pAllocationCallbacks, VMA_NULL };
6674	if (!isVirutal && IsEnabled())
6675	{
6676	ctx.pageAllocs = vma_new_array(pAllocationCallbacks, uint16_t, m_RegionCount);
6677	memset(ctx.pageAllocs, `0`, m_RegionCount * sizeof(uint16_t));
6678	}
6679	return ctx;
6680	}
6681
6682	bool VmaBlockBufferImageGranularity::Validate(ValidationContext& ctx,
6683	VkDeviceSize offset, VkDeviceSize size) const
6684	{
6685	if (IsEnabled())
6686	{
6687	uint32_t start = GetStartPage(offset);
6688	++ctx.pageAllocs[start];
6689	VMA_VALIDATE(m_RegionInfo[start].allocCount > `0`);
6690
6691	uint32_t end = GetEndPage(offset, size);
6692	if (start != end)
6693	{
6694	++ctx.pageAllocs[end];
6695	VMA_VALIDATE(m_RegionInfo[end].allocCount > `0`);
6696	}
6697	}
6698	return true;
6699	}
6700
6701	bool VmaBlockBufferImageGranularity::FinishValidation(ValidationContext& ctx) const
6702	{
6703	// Check proper page structure
6704	if (IsEnabled())
6705	{
6706	VMA_ASSERT(ctx.pageAllocs != VMA_NULL && "Validation context not initialized!");
6707
6708	for (uint32_t page = `0`; page < m_RegionCount; ++page)
6709	{
6710	VMA_VALIDATE(ctx.pageAllocs[page] == m_RegionInfo[page].allocCount);
6711	}
6712	vma_delete_array(ctx.allocCallbacks, ctx.pageAllocs, m_RegionCount);
6713	ctx.pageAllocs = VMA_NULL;
6714	}
6715	return true;
6716	}
6717
6718	uint32_t VmaBlockBufferImageGranularity::OffsetToPageIndex(VkDeviceSize offset) const
6719	{
6720	return static_cast<uint32_t>(offset >> VMA_BITSCAN_MSB(m_BufferImageGranularity));
6721	}
6722
6723	void VmaBlockBufferImageGranularity::AllocPage(RegionInfo& page, uint8_t allocType)
6724	{
6725	// When current alloc type is free then it can be overriden by new type
6726	if (page.allocCount == `0` \|\| (page.allocCount > `0` && page.allocType == VMA_SUBALLOCATION_TYPE_FREE))
6727	page.allocType = allocType;
6728
6729	++page.allocCount;
6730	}
6731	#endif // _VMA_BLOCK_BUFFER_IMAGE_GRANULARITY_FUNCTIONS
6732	#endif // _VMA_BLOCK_BUFFER_IMAGE_GRANULARITY
6733
6734	#if 0
6735	#ifndef _VMA_BLOCK_METADATA_GENERIC
6736	class VmaBlockMetadata_Generic : public VmaBlockMetadata
6737	{
6738	friend class VmaDefragmentationAlgorithm_Generic;
6739	friend class VmaDefragmentationAlgorithm_Fast;
6740	VMA_CLASS_NO_COPY(VmaBlockMetadata_Generic)
6741	public:
6742	VmaBlockMetadata_Generic(const VkAllocationCallbacks* pAllocationCallbacks,
6743	VkDeviceSize bufferImageGranularity, bool isVirtual);
6744	virtual ~VmaBlockMetadata_Generic() = default;
6745
6746	size_t GetAllocationCount() const override { return m_Suballocations.size() - m_FreeCount; }
6747	VkDeviceSize GetSumFreeSize() const override { return m_SumFreeSize; }
6748	bool IsEmpty() const override { return (m_Suballocations.size() == `1`) && (m_FreeCount == `1`); }
6749	void Free(VmaAllocHandle allocHandle) override { FreeSuballocation(FindAtOffset((VkDeviceSize)allocHandle - `1`)); }
6750	VkDeviceSize GetAllocationOffset(VmaAllocHandle allocHandle) const override { return (VkDeviceSize)allocHandle - `1`; };
6751
6752	void Init(VkDeviceSize size) override;
6753	bool Validate() const override;
6754
6755	void AddDetailedStatistics(VmaDetailedStatistics& inoutStats) const override;
6756	void AddStatistics(VmaStatistics& inoutStats) const override;
6757
6758	#if VMA_STATS_STRING_ENABLED
6759	void PrintDetailedMap(class VmaJsonWriter& json, uint32_t mapRefCount) const override;
6760	#endif
6761
6762	bool CreateAllocationRequest(
6763	VkDeviceSize allocSize,
6764	VkDeviceSize allocAlignment,
6765	bool upperAddress,
6766	VmaSuballocationType allocType,
6767	uint32_t strategy,
6768	VmaAllocationRequest* pAllocationRequest) override;
6769
6770	VkResult CheckCorruption(const void* pBlockData) override;
6771
6772	void Alloc(
6773	const VmaAllocationRequest& request,
6774	VmaSuballocationType type,
6775	void* userData) override;
6776
6777	void GetAllocationInfo(VmaAllocHandle allocHandle, VmaVirtualAllocationInfo& outInfo) override;
6778	void* GetAllocationUserData(VmaAllocHandle allocHandle) const override;
6779	VmaAllocHandle GetAllocationListBegin() const override;
6780	VmaAllocHandle GetNextAllocation(VmaAllocHandle prevAlloc) const override;
6781	void Clear() override;
6782	void SetAllocationUserData(VmaAllocHandle allocHandle, void* userData) override;
6783	void DebugLogAllAllocations() const override;
6784
6785	private:
6786	uint32_t m_FreeCount;
6787	VkDeviceSize m_SumFreeSize;
6788	VmaSuballocationList m_Suballocations;
6789	// Suballocations that are free. Sorted by size, ascending.
6790	VmaVector<VmaSuballocationList::iterator, VmaStlAllocator<VmaSuballocationList::iterator>> m_FreeSuballocationsBySize;
6791
6792	VkDeviceSize AlignAllocationSize(VkDeviceSize size) const { return IsVirtual() ? size : VmaAlignUp(size, (VkDeviceSize)`16`); }
6793
6794	VmaSuballocationList::iterator FindAtOffset(VkDeviceSize offset) const;
6795	bool ValidateFreeSuballocationList() const;
6796
6797	// Checks if requested suballocation with given parameters can be placed in given pFreeSuballocItem.
6798	// If yes, fills pOffset and returns true. If no, returns false.
6799	bool CheckAllocation(
6800	VkDeviceSize allocSize,
6801	VkDeviceSize allocAlignment,
6802	VmaSuballocationType allocType,
6803	VmaSuballocationList::const_iterator suballocItem,
6804	VmaAllocHandle* pAllocHandle) const;
6805
6806	// Given free suballocation, it merges it with following one, which must also be free.
6807	void MergeFreeWithNext(VmaSuballocationList::iterator item);
6808	// Releases given suballocation, making it free.
6809	// Merges it with adjacent free suballocations if applicable.
6810	// Returns iterator to new free suballocation at this place.
6811	VmaSuballocationList::iterator FreeSuballocation(VmaSuballocationList::iterator suballocItem);
6812	// Given free suballocation, it inserts it into sorted list of
6813	// m_FreeSuballocationsBySize if it is suitable.
6814	void RegisterFreeSuballocation(VmaSuballocationList::iterator item);
6815	// Given free suballocation, it removes it from sorted list of
6816	// m_FreeSuballocationsBySize if it is suitable.
6817	void UnregisterFreeSuballocation(VmaSuballocationList::iterator item);
6818	};
6819
6820	#ifndef _VMA_BLOCK_METADATA_GENERIC_FUNCTIONS
6821	VmaBlockMetadata_Generic::VmaBlockMetadata_Generic(const VkAllocationCallbacks* pAllocationCallbacks,
6822	VkDeviceSize bufferImageGranularity, bool isVirtual)
6823	: VmaBlockMetadata(pAllocationCallbacks, bufferImageGranularity, isVirtual),
6824	m_FreeCount(`0`),
6825	m_SumFreeSize(`0`),
6826	m_Suballocations(VmaStlAllocator<VmaSuballocation>(pAllocationCallbacks)),
6827	m_FreeSuballocationsBySize(VmaStlAllocator<VmaSuballocationList::iterator>(pAllocationCallbacks)) {}
6828
6829	void VmaBlockMetadata_Generic::Init(VkDeviceSize size)
6830	{
6831	VmaBlockMetadata::Init(size);
6832
6833	m_FreeCount = `1`;
6834	m_SumFreeSize = size;
6835
6836	VmaSuballocation suballoc = {};
6837	suballoc.offset = `0`;
6838	suballoc.size = size;
6839	suballoc.type = VMA_SUBALLOCATION_TYPE_FREE;
6840
6841	m_Suballocations.push_back(suballoc);
6842	m_FreeSuballocationsBySize.push_back(m_Suballocations.begin());
6843	}
6844
6845	bool VmaBlockMetadata_Generic::Validate() const
6846	{
6847	VMA_VALIDATE(!m_Suballocations.empty());
6848
6849	// Expected offset of new suballocation as calculated from previous ones.
6850	VkDeviceSize calculatedOffset = `0`;
6851	// Expected number of free suballocations as calculated from traversing their list.
6852	uint32_t calculatedFreeCount = `0`;
6853	// Expected sum size of free suballocations as calculated from traversing their list.
6854	VkDeviceSize calculatedSumFreeSize = `0`;
6855	// Expected number of free suballocations that should be registered in
6856	// m_FreeSuballocationsBySize calculated from traversing their list.
6857	size_t freeSuballocationsToRegister = `0`;
6858	// True if previous visited suballocation was free.
6859	bool prevFree = false;
6860
6861	const VkDeviceSize debugMargin = GetDebugMargin();
6862
6863	for (const auto& subAlloc : m_Suballocations)
6864	{
6865	// Actual offset of this suballocation doesn't match expected one.
6866	VMA_VALIDATE(subAlloc.offset == calculatedOffset);
6867
6868	const bool currFree = (subAlloc.type == VMA_SUBALLOCATION_TYPE_FREE);
6869	// Two adjacent free suballocations are invalid. They should be merged.
6870	VMA_VALIDATE(!prevFree \|\| !currFree);
6871
6872	VmaAllocation alloc = (VmaAllocation)subAlloc.userData;
6873	if (!IsVirtual())
6874	{
6875	VMA_VALIDATE(currFree == (alloc == VK_NULL_HANDLE));
6876	}
6877
6878	if (currFree)
6879	{
6880	calculatedSumFreeSize += subAlloc.size;
6881	++calculatedFreeCount;
6882	++freeSuballocationsToRegister;
6883
6884	// Margin required between allocations - every free space must be at least that large.
6885	VMA_VALIDATE(subAlloc.size >= debugMargin);
6886	}
6887	else
6888	{
6889	if (!IsVirtual())
6890	{
6891	VMA_VALIDATE((VkDeviceSize)alloc->GetAllocHandle() == subAlloc.offset + `1`);
6892	VMA_VALIDATE(alloc->GetSize() == subAlloc.size);
6893	}
6894
6895	// Margin required between allocations - previous allocation must be free.
6896	VMA_VALIDATE(debugMargin == `0` \|\| prevFree);
6897	}
6898
6899	calculatedOffset += subAlloc.size;
6900	prevFree = currFree;
6901	}
6902
6903	// Number of free suballocations registered in m_FreeSuballocationsBySize doesn't
6904	// match expected one.
6905	VMA_VALIDATE(m_FreeSuballocationsBySize.size() == freeSuballocationsToRegister);
6906
6907	VkDeviceSize lastSize = `0`;
6908	for (size_t i = `0`; i < m_FreeSuballocationsBySize.size(); ++i)
6909	{
6910	VmaSuballocationList::iterator suballocItem = m_FreeSuballocationsBySize[i];
6911
6912	// Only free suballocations can be registered in m_FreeSuballocationsBySize.
6913	VMA_VALIDATE(suballocItem->type == VMA_SUBALLOCATION_TYPE_FREE);
6914	// They must be sorted by size ascending.
6915	VMA_VALIDATE(suballocItem->size >= lastSize);
6916
6917	lastSize = suballocItem->size;
6918	}
6919
6920	// Check if totals match calculated values.
6921	VMA_VALIDATE(ValidateFreeSuballocationList());
6922	VMA_VALIDATE(calculatedOffset == GetSize());
6923	VMA_VALIDATE(calculatedSumFreeSize == m_SumFreeSize);
6924	VMA_VALIDATE(calculatedFreeCount == m_FreeCount);
6925
6926	return true;
6927	}
6928
6929	void VmaBlockMetadata_Generic::AddDetailedStatistics(VmaDetailedStatistics& inoutStats) const
6930	{
6931	const uint32_t rangeCount = (uint32_t)m_Suballocations.size();
6932	inoutStats.statistics.blockCount++;
6933	inoutStats.statistics.blockBytes += GetSize();
6934
6935	for (const auto& suballoc : m_Suballocations)
6936	{
6937	if (suballoc.type != VMA_SUBALLOCATION_TYPE_FREE)
6938	VmaAddDetailedStatisticsAllocation(inoutStats, suballoc.size);
6939	else
6940	VmaAddDetailedStatisticsUnusedRange(inoutStats, suballoc.size);
6941	}
6942	}
6943
6944	void VmaBlockMetadata_Generic::AddStatistics(VmaStatistics& inoutStats) const
6945	{
6946	inoutStats.blockCount++;
6947	inoutStats.allocationCount += (uint32_t)m_Suballocations.size() - m_FreeCount;
6948	inoutStats.blockBytes += GetSize();
6949	inoutStats.allocationBytes += GetSize() - m_SumFreeSize;
6950	}
6951
6952	#if VMA_STATS_STRING_ENABLED
6953	void VmaBlockMetadata_Generic::PrintDetailedMap(class VmaJsonWriter& json, uint32_t mapRefCount) const
6954	{
6955	PrintDetailedMap_Begin(json,
6956	m_SumFreeSize, // unusedBytes
6957	m_Suballocations.size() - (size_t)m_FreeCount, // allocationCount
6958	m_FreeCount, // unusedRangeCount
6959	mapRefCount);
6960
6961	for (const auto& suballoc : m_Suballocations)
6962	{
6963	if (suballoc.type == VMA_SUBALLOCATION_TYPE_FREE)
6964	{
6965	PrintDetailedMap_UnusedRange(json, suballoc.offset, suballoc.size);
6966	}
6967	else
6968	{
6969	PrintDetailedMap_Allocation(json, suballoc.offset, suballoc.size, suballoc.userData);
6970	}
6971	}
6972
6973	PrintDetailedMap_End(json);
6974	}
6975	#endif // VMA_STATS_STRING_ENABLED
6976
6977	bool VmaBlockMetadata_Generic::CreateAllocationRequest(
6978	VkDeviceSize allocSize,
6979	VkDeviceSize allocAlignment,
6980	bool upperAddress,
6981	VmaSuballocationType allocType,
6982	uint32_t strategy,
6983	VmaAllocationRequest* pAllocationRequest)
6984	{
6985	VMA_ASSERT(allocSize > `0`);
6986	VMA_ASSERT(!upperAddress);
6987	VMA_ASSERT(allocType != VMA_SUBALLOCATION_TYPE_FREE);
6988	VMA_ASSERT(pAllocationRequest != VMA_NULL);
6989	VMA_HEAVY_ASSERT(Validate());
6990
6991	allocSize = AlignAllocationSize(allocSize);
6992
6993	pAllocationRequest->type = VmaAllocationRequestType::Normal;
6994	pAllocationRequest->size = allocSize;
6995
6996	const VkDeviceSize debugMargin = GetDebugMargin();
6997
6998	// There is not enough total free space in this block to fulfill the request: Early return.
6999	if (m_SumFreeSize < allocSize + debugMargin)
7000	{
7001	return false;
7002	}
7003
7004	// New algorithm, efficiently searching freeSuballocationsBySize.
7005	const size_t freeSuballocCount = m_FreeSuballocationsBySize.size();
7006	if (freeSuballocCount > `0`)
7007	{
7008	if (strategy == `0` \|\|
7009	strategy == VMA_ALLOCATION_CREATE_STRATEGY_MIN_MEMORY_BIT)
7010	{
7011	// Find first free suballocation with size not less than allocSize + debugMargin.
7012	VmaSuballocationList::iterator* const it = VmaBinaryFindFirstNotLess(
7013	m_FreeSuballocationsBySize.data(),
7014	m_FreeSuballocationsBySize.data() + freeSuballocCount,
7015	allocSize + debugMargin,
7016	VmaSuballocationItemSizeLess());
7017	size_t index = it - m_FreeSuballocationsBySize.data();
7018	for (; index < freeSuballocCount; ++index)
7019	{
7020	if (CheckAllocation(
7021	allocSize,
7022	allocAlignment,
7023	allocType,
7024	m_FreeSuballocationsBySize[index],
7025	&pAllocationRequest->allocHandle))
7026	{
7027	pAllocationRequest->item = m_FreeSuballocationsBySize[index];
7028	return true;
7029	}
7030	}
7031	}
7032	else if (strategy == VMA_ALLOCATION_INTERNAL_STRATEGY_MIN_OFFSET)
7033	{
7034	for (VmaSuballocationList::iterator it = m_Suballocations.begin();
7035	it != m_Suballocations.end();
7036	++it)
7037	{
7038	if (it->type == VMA_SUBALLOCATION_TYPE_FREE && CheckAllocation(
7039	allocSize,
7040	allocAlignment,
7041	allocType,
7042	it,
7043	&pAllocationRequest->allocHandle))
7044	{
7045	pAllocationRequest->item = it;
7046	return true;
7047	}
7048	}
7049	}
7050	else
7051	{
7052	VMA_ASSERT(strategy & (VMA_ALLOCATION_CREATE_STRATEGY_MIN_TIME_BIT \| VMA_ALLOCATION_CREATE_STRATEGY_MIN_OFFSET_BIT ));
7053	// Search staring from biggest suballocations.
7054	for (size_t index = freeSuballocCount; index--; )
7055	{
7056	if (CheckAllocation(
7057	allocSize,
7058	allocAlignment,
7059	allocType,
7060	m_FreeSuballocationsBySize[index],
7061	&pAllocationRequest->allocHandle))
7062	{
7063	pAllocationRequest->item = m_FreeSuballocationsBySize[index];
7064	return true;
7065	}
7066	}
7067	}
7068	}
7069
7070	return false;
7071	}
7072
7073	VkResult VmaBlockMetadata_Generic::CheckCorruption(const void* pBlockData)
7074	{
7075	for (auto& suballoc : m_Suballocations)
7076	{
7077	if (suballoc.type != VMA_SUBALLOCATION_TYPE_FREE)
7078	{
7079	if (!VmaValidateMagicValue(pBlockData, suballoc.offset + suballoc.size))
7080	{
7081	VMA_ASSERT(`0` && "MEMORY CORRUPTION DETECTED AFTER VALIDATED ALLOCATION!");
7082	return VK_ERROR_UNKNOWN_COPY;
7083	}
7084	}
7085	}
7086
7087	return VK_SUCCESS;
7088	}
7089
7090	void VmaBlockMetadata_Generic::Alloc(
7091	const VmaAllocationRequest& request,
7092	VmaSuballocationType type,
7093	void* userData)
7094	{
7095	VMA_ASSERT(request.type == VmaAllocationRequestType::Normal);
7096	VMA_ASSERT(request.item != m_Suballocations.end());
7097	VmaSuballocation& suballoc = *request.item;
7098	// Given suballocation is a free block.
7099	VMA_ASSERT(suballoc.type == VMA_SUBALLOCATION_TYPE_FREE);
7100
7101	// Given offset is inside this suballocation.
7102	VMA_ASSERT((VkDeviceSize)request.allocHandle - `1` >= suballoc.offset);
7103	const VkDeviceSize paddingBegin = (VkDeviceSize)request.allocHandle - suballoc.offset - `1`;
7104	VMA_ASSERT(suballoc.size >= paddingBegin + request.size);
7105	const VkDeviceSize paddingEnd = suballoc.size - paddingBegin - request.size;
7106
7107	// Unregister this free suballocation from m_FreeSuballocationsBySize and update
7108	// it to become used.
7109	UnregisterFreeSuballocation(request.item);
7110
7111	suballoc.offset = (VkDeviceSize)request.allocHandle - `1`;
7112	suballoc.size = request.size;
7113	suballoc.type = type;
7114	suballoc.userData = userData;
7115
7116	// If there are any free bytes remaining at the end, insert new free suballocation after current one.
7117	if (paddingEnd)
7118	{
7119	VmaSuballocation paddingSuballoc = {};
7120	paddingSuballoc.offset = suballoc.offset + suballoc.size;
7121	paddingSuballoc.size = paddingEnd;
7122	paddingSuballoc.type = VMA_SUBALLOCATION_TYPE_FREE;
7123	VmaSuballocationList::iterator next = request.item;
7124	++next;
7125	const VmaSuballocationList::iterator paddingEndItem =
7126	m_Suballocations.insert(next, paddingSuballoc);
7127	RegisterFreeSuballocation(paddingEndItem);
7128	}
7129
7130	// If there are any free bytes remaining at the beginning, insert new free suballocation before current one.
7131	if (paddingBegin)
7132	{
7133	VmaSuballocation paddingSuballoc = {};
7134	paddingSuballoc.offset = suballoc.offset - paddingBegin;
7135	paddingSuballoc.size = paddingBegin;
7136	paddingSuballoc.type = VMA_SUBALLOCATION_TYPE_FREE;
7137	const VmaSuballocationList::iterator paddingBeginItem =
7138	m_Suballocations.insert(request.item, paddingSuballoc);
7139	RegisterFreeSuballocation(paddingBeginItem);
7140	}
7141
7142	// Update totals.
7143	m_FreeCount = m_FreeCount - `1`;
7144	if (paddingBegin > `0`)
7145	{
7146	++m_FreeCount;
7147	}
7148	if (paddingEnd > `0`)
7149	{
7150	++m_FreeCount;
7151	}
7152	m_SumFreeSize -= request.size;
7153	}
7154
7155	void VmaBlockMetadata_Generic::GetAllocationInfo(VmaAllocHandle allocHandle, VmaVirtualAllocationInfo& outInfo)
7156	{
7157	outInfo.offset = (VkDeviceSize)allocHandle - `1`;
7158	const VmaSuballocation& suballoc = *FindAtOffset(outInfo.offset);
7159	outInfo.size = suballoc.size;
7160	outInfo.pUserData = suballoc.userData;
7161	}
7162
7163	void* VmaBlockMetadata_Generic::GetAllocationUserData(VmaAllocHandle allocHandle) const
7164	{
7165	return FindAtOffset((VkDeviceSize)allocHandle - `1`)->userData;
7166	}
7167
7168	VmaAllocHandle VmaBlockMetadata_Generic::GetAllocationListBegin() const
7169	{
7170	if (IsEmpty())
7171	return VK_NULL_HANDLE;
7172
7173	for (const auto& suballoc : m_Suballocations)
7174	{
7175	if (suballoc.type != VMA_SUBALLOCATION_TYPE_FREE)
7176	return (VmaAllocHandle)(suballoc.offset + `1`);
7177	}
7178	VMA_ASSERT(false && "Should contain at least 1 allocation!");
7179	return VK_NULL_HANDLE;
7180	}
7181
7182	VmaAllocHandle VmaBlockMetadata_Generic::GetNextAllocation(VmaAllocHandle prevAlloc) const
7183	{
7184	VmaSuballocationList::const_iterator prev = FindAtOffset((VkDeviceSize)prevAlloc - `1`);
7185
7186	for (VmaSuballocationList::const_iterator it = ++prev; it != m_Suballocations.end(); ++it)
7187	{
7188	if (it->type != VMA_SUBALLOCATION_TYPE_FREE)
7189	return (VmaAllocHandle)(it->offset + `1`);
7190	}
7191	return VK_NULL_HANDLE;
7192	}
7193
7194	void VmaBlockMetadata_Generic::Clear()
7195	{
7196	const VkDeviceSize size = GetSize();
7197
7198	VMA_ASSERT(IsVirtual());
7199	m_FreeCount = `1`;
7200	m_SumFreeSize = size;
7201	m_Suballocations.clear();
7202	m_FreeSuballocationsBySize.clear();
7203
7204	VmaSuballocation suballoc = {};
7205	suballoc.offset = `0`;
7206	suballoc.size = size;
7207	suballoc.type = VMA_SUBALLOCATION_TYPE_FREE;
7208	m_Suballocations.push_back(suballoc);
7209
7210	m_FreeSuballocationsBySize.push_back(m_Suballocations.begin());
7211	}
7212
7213	void VmaBlockMetadata_Generic::SetAllocationUserData(VmaAllocHandle allocHandle, void* userData)
7214	{
7215	VmaSuballocation& suballoc = *FindAtOffset((VkDeviceSize)allocHandle - `1`);
7216	suballoc.userData = userData;
7217	}
7218
7219	void VmaBlockMetadata_Generic::DebugLogAllAllocations() const
7220	{
7221	for (const auto& suballoc : m_Suballocations)
7222	{
7223	if (suballoc.type != VMA_SUBALLOCATION_TYPE_FREE)
7224	DebugLogAllocation(suballoc.offset, suballoc.size, suballoc.userData);
7225	}
7226	}
7227
7228	VmaSuballocationList::iterator VmaBlockMetadata_Generic::FindAtOffset(VkDeviceSize offset) const
7229	{
7230	VMA_HEAVY_ASSERT(!m_Suballocations.empty());
7231	const VkDeviceSize last = m_Suballocations.rbegin()->offset;
7232	if (last == offset)
7233	return m_Suballocations.rbegin().drop_const();
7234	const VkDeviceSize first = m_Suballocations.begin()->offset;
7235	if (first == offset)
7236	return m_Suballocations.begin().drop_const();
7237
7238	const size_t suballocCount = m_Suballocations.size();
7239	const VkDeviceSize step = (last - first + m_Suballocations.begin()->size) / suballocCount;
7240	auto findSuballocation = [&](auto begin, auto end) -> VmaSuballocationList::iterator
7241	{
7242	for (auto suballocItem = begin;
7243	suballocItem != end;
7244	++suballocItem)
7245	{
7246	if (suballocItem->offset == offset)
7247	return suballocItem.drop_const();
7248	}
7249	VMA_ASSERT(false && "Not found!");
7250	return m_Suballocations.end().drop_const();
7251	};
7252	// If requested offset is closer to the end of range, search from the end
7253	if (offset - first > suballocCount * step / `2`)
7254	{
7255	return findSuballocation(m_Suballocations.rbegin(), m_Suballocations.rend());
7256	}
7257	return findSuballocation(m_Suballocations.begin(), m_Suballocations.end());
7258	}
7259
7260	bool VmaBlockMetadata_Generic::ValidateFreeSuballocationList() const
7261	{
7262	VkDeviceSize lastSize = `0`;
7263	for (size_t i = `0`, count = m_FreeSuballocationsBySize.size(); i < count; ++i)
7264	{
7265	const VmaSuballocationList::iterator it = m_FreeSuballocationsBySize[i];
7266
7267	VMA_VALIDATE(it->type == VMA_SUBALLOCATION_TYPE_FREE);
7268	VMA_VALIDATE(it->size >= lastSize);
7269	lastSize = it->size;
7270	}
7271	return true;
7272	}
7273
7274	bool VmaBlockMetadata_Generic::CheckAllocation(
7275	VkDeviceSize allocSize,
7276	VkDeviceSize allocAlignment,
7277	VmaSuballocationType allocType,
7278	VmaSuballocationList::const_iterator suballocItem,
7279	VmaAllocHandle* pAllocHandle) const
7280	{
7281	VMA_ASSERT(allocSize > `0`);
7282	VMA_ASSERT(allocType != VMA_SUBALLOCATION_TYPE_FREE);
7283	VMA_ASSERT(suballocItem != m_Suballocations.cend());
7284	VMA_ASSERT(pAllocHandle != VMA_NULL);
7285
7286	const VkDeviceSize debugMargin = GetDebugMargin();
7287	const VkDeviceSize bufferImageGranularity = GetBufferImageGranularity();
7288
7289	const VmaSuballocation& suballoc = *suballocItem;
7290	VMA_ASSERT(suballoc.type == VMA_SUBALLOCATION_TYPE_FREE);
7291
7292	// Size of this suballocation is too small for this request: Early return.
7293	if (suballoc.size < allocSize)
7294	{
7295	return false;
7296	}
7297
7298	// Start from offset equal to beginning of this suballocation.
7299	VkDeviceSize offset = suballoc.offset + (suballocItem == m_Suballocations.cbegin() ? `0` : GetDebugMargin());
7300
7301	// Apply debugMargin from the end of previous alloc.
7302	if (debugMargin > `0`)
7303	{
7304	offset += debugMargin;
7305	}
7306
7307	// Apply alignment.
7308	offset = VmaAlignUp(offset, allocAlignment);
7309
7310	// Check previous suballocations for BufferImageGranularity conflicts.
7311	// Make bigger alignment if necessary.
7312	if (bufferImageGranularity > `1` && bufferImageGranularity != allocAlignment)
7313	{
7314	bool bufferImageGranularityConflict = false;
7315	VmaSuballocationList::const_iterator prevSuballocItem = suballocItem;
7316	while (prevSuballocItem != m_Suballocations.cbegin())
7317	{
7318	--prevSuballocItem;
7319	const VmaSuballocation& prevSuballoc = *prevSuballocItem;
7320	if (VmaBlocksOnSamePage(prevSuballoc.offset, prevSuballoc.size, offset, bufferImageGranularity))
7321	{
7322	if (VmaIsBufferImageGranularityConflict(prevSuballoc.type, allocType))
7323	{
7324	bufferImageGranularityConflict = true;
7325	break;
7326	}
7327	}
7328	else
7329	// Already on previous page.
7330	break;
7331	}
7332	if (bufferImageGranularityConflict)
7333	{
7334	offset = VmaAlignUp(offset, bufferImageGranularity);
7335	}
7336	}
7337
7338	// Calculate padding at the beginning based on current offset.
7339	const VkDeviceSize paddingBegin = offset - suballoc.offset;
7340
7341	// Fail if requested size plus margin after is bigger than size of this suballocation.
7342	if (paddingBegin + allocSize + debugMargin > suballoc.size)
7343	{
7344	return false;
7345	}
7346
7347	// Check next suballocations for BufferImageGranularity conflicts.
7348	// If conflict exists, allocation cannot be made here.
7349	if (allocSize % bufferImageGranularity \|\| offset % bufferImageGranularity)
7350	{
7351	VmaSuballocationList::const_iterator nextSuballocItem = suballocItem;
7352	++nextSuballocItem;
7353	while (nextSuballocItem != m_Suballocations.cend())
7354	{
7355	const VmaSuballocation& nextSuballoc = *nextSuballocItem;
7356	if (VmaBlocksOnSamePage(offset, allocSize, nextSuballoc.offset, bufferImageGranularity))
7357	{
7358	if (VmaIsBufferImageGranularityConflict(allocType, nextSuballoc.type))
7359	{
7360	return false;
7361	}
7362	}
7363	else
7364	{
7365	// Already on next page.
7366	break;
7367	}
7368	++nextSuballocItem;
7369	}
7370	}
7371
7372	*pAllocHandle = (VmaAllocHandle)(offset + `1`);
7373	// All tests passed: Success. pAllocHandle is already filled.
7374	return true;
7375	}
7376
7377	void VmaBlockMetadata_Generic::MergeFreeWithNext(VmaSuballocationList::iterator item)
7378	{
7379	VMA_ASSERT(item != m_Suballocations.end());
7380	VMA_ASSERT(item->type == VMA_SUBALLOCATION_TYPE_FREE);
7381
7382	VmaSuballocationList::iterator nextItem = item;
7383	++nextItem;
7384	VMA_ASSERT(nextItem != m_Suballocations.end());
7385	VMA_ASSERT(nextItem->type == VMA_SUBALLOCATION_TYPE_FREE);
7386
7387	item->size += nextItem->size;
7388	--m_FreeCount;
7389	m_Suballocations.erase(nextItem);
7390	}
7391
7392	VmaSuballocationList::iterator VmaBlockMetadata_Generic::FreeSuballocation(VmaSuballocationList::iterator suballocItem)
7393	{
7394	// Change this suballocation to be marked as free.
7395	VmaSuballocation& suballoc = *suballocItem;
7396	suballoc.type = VMA_SUBALLOCATION_TYPE_FREE;
7397	suballoc.userData = VMA_NULL;
7398
7399	// Update totals.
7400	++m_FreeCount;
7401	m_SumFreeSize += suballoc.size;
7402
7403	// Merge with previous and/or next suballocation if it's also free.
7404	bool mergeWithNext = false;
7405	bool mergeWithPrev = false;
7406
7407	VmaSuballocationList::iterator nextItem = suballocItem;
7408	++nextItem;
7409	if ((nextItem != m_Suballocations.end()) && (nextItem->type == VMA_SUBALLOCATION_TYPE_FREE))
7410	{
7411	mergeWithNext = true;
7412	}
7413
7414	VmaSuballocationList::iterator prevItem = suballocItem;
7415	if (suballocItem != m_Suballocations.begin())
7416	{
7417	--prevItem;
7418	if (prevItem->type == VMA_SUBALLOCATION_TYPE_FREE)
7419	{
7420	mergeWithPrev = true;
7421	}
7422	}
7423
7424	if (mergeWithNext)
7425	{
7426	UnregisterFreeSuballocation(nextItem);
7427	MergeFreeWithNext(suballocItem);
7428	}
7429
7430	if (mergeWithPrev)
7431	{
7432	UnregisterFreeSuballocation(prevItem);
7433	MergeFreeWithNext(prevItem);
7434	RegisterFreeSuballocation(prevItem);
7435	return prevItem;
7436	}
7437	else
7438	{
7439	RegisterFreeSuballocation(suballocItem);
7440	return suballocItem;
7441	}
7442	}
7443
7444	void VmaBlockMetadata_Generic::RegisterFreeSuballocation(VmaSuballocationList::iterator item)
7445	{
7446	VMA_ASSERT(item->type == VMA_SUBALLOCATION_TYPE_FREE);
7447	VMA_ASSERT(item->size > `0`);
7448
7449	// You may want to enable this validation at the beginning or at the end of
7450	// this function, depending on what do you want to check.
7451	VMA_HEAVY_ASSERT(ValidateFreeSuballocationList());
7452
7453	if (m_FreeSuballocationsBySize.empty())
7454	{
7455	m_FreeSuballocationsBySize.push_back(item);
7456	}
7457	else
7458	{
7459	VmaVectorInsertSorted<VmaSuballocationItemSizeLess>(m_FreeSuballocationsBySize, item);
7460	}
7461
7462	//VMA_HEAVY_ASSERT(ValidateFreeSuballocationList());
7463	}
7464
7465	void VmaBlockMetadata_Generic::UnregisterFreeSuballocation(VmaSuballocationList::iterator item)
7466	{
7467	VMA_ASSERT(item->type == VMA_SUBALLOCATION_TYPE_FREE);
7468	VMA_ASSERT(item->size > `0`);
7469
7470	// You may want to enable this validation at the beginning or at the end of
7471	// this function, depending on what do you want to check.
7472	VMA_HEAVY_ASSERT(ValidateFreeSuballocationList());
7473
7474	VmaSuballocationList::iterator* const it = VmaBinaryFindFirstNotLess(
7475	m_FreeSuballocationsBySize.data(),
7476	m_FreeSuballocationsBySize.data() + m_FreeSuballocationsBySize.size(),
7477	item,
7478	VmaSuballocationItemSizeLess());
7479	for (size_t index = it - m_FreeSuballocationsBySize.data();
7480	index < m_FreeSuballocationsBySize.size();
7481	++index)
7482	{
7483	if (m_FreeSuballocationsBySize[index] == item)
7484	{
7485	VmaVectorRemove(m_FreeSuballocationsBySize, index);
7486	return;
7487	}
7488	VMA_ASSERT((m_FreeSuballocationsBySize[index]->size == item->size) && "Not found.");
7489	}
7490	VMA_ASSERT(`0` && "Not found.");
7491
7492	//VMA_HEAVY_ASSERT(ValidateFreeSuballocationList());
7493	}
7494	#endif // _VMA_BLOCK_METADATA_GENERIC_FUNCTIONS
7495	#endif // _VMA_BLOCK_METADATA_GENERIC
7496	#endif // #if 0
7497
7498	#ifndef _VMA_BLOCK_METADATA_LINEAR
7499	/*
7500	Allocations and their references in internal data structure look like this:
7501
7502	if(m_2ndVectorMode == SECOND_VECTOR_EMPTY):
7503
7504	0 +-------+
7505	\| \|
7506	\| \|
7507	\| \|
7508	+-------+
7509	\| Alloc \| 1st[m_1stNullItemsBeginCount]
7510	+-------+
7511	\| Alloc \| 1st[m_1stNullItemsBeginCount + 1]
7512	+-------+
7513	\| ... \|
7514	+-------+
7515	\| Alloc \| 1st[1st.size() - 1]
7516	+-------+
7517	\| \|
7518	\| \|
7519	\| \|
7520	GetSize() +-------+
7521
7522	if(m_2ndVectorMode == SECOND_VECTOR_RING_BUFFER):
7523
7524	0 +-------+
7525	\| Alloc \| 2nd[0]
7526	+-------+
7527	\| Alloc \| 2nd[1]
7528	+-------+
7529	\| ... \|
7530	+-------+
7531	\| Alloc \| 2nd[2nd.size() - 1]
7532	+-------+
7533	\| \|
7534	\| \|
7535	\| \|
7536	+-------+
7537	\| Alloc \| 1st[m_1stNullItemsBeginCount]
7538	+-------+
7539	\| Alloc \| 1st[m_1stNullItemsBeginCount + 1]
7540	+-------+
7541	\| ... \|
7542	+-------+
7543	\| Alloc \| 1st[1st.size() - 1]
7544	+-------+
7545	\| \|
7546	GetSize() +-------+
7547
7548	if(m_2ndVectorMode == SECOND_VECTOR_DOUBLE_STACK):
7549
7550	0 +-------+
7551	\| \|
7552	\| \|
7553	\| \|
7554	+-------+
7555	\| Alloc \| 1st[m_1stNullItemsBeginCount]
7556	+-------+
7557	\| Alloc \| 1st[m_1stNullItemsBeginCount + 1]
7558	+-------+
7559	\| ... \|
7560	+-------+
7561	\| Alloc \| 1st[1st.size() - 1]
7562	+-------+
7563	\| \|
7564	\| \|
7565	\| \|
7566	+-------+
7567	\| Alloc \| 2nd[2nd.size() - 1]
7568	+-------+
7569	\| ... \|
7570	+-------+
7571	\| Alloc \| 2nd[1]
7572	+-------+
7573	\| Alloc \| 2nd[0]
7574	GetSize() +-------+
7575
7576	*/
7577	class VmaBlockMetadata_Linear : public VmaBlockMetadata
7578	{
7579	VMA_CLASS_NO_COPY(VmaBlockMetadata_Linear)
7580	public:
7581	VmaBlockMetadata_Linear(const VkAllocationCallbacks* pAllocationCallbacks,
7582	VkDeviceSize bufferImageGranularity, bool isVirtual);
7583	virtual ~VmaBlockMetadata_Linear() = default;
7584
7585	VkDeviceSize GetSumFreeSize() const override { return m_SumFreeSize; }
7586	bool IsEmpty() const override { return GetAllocationCount() == `0`; }
7587	VkDeviceSize GetAllocationOffset(VmaAllocHandle allocHandle) const override { return (VkDeviceSize)allocHandle - `1`; };
7588
7589	void Init(VkDeviceSize size) override;
7590	bool Validate() const override;
7591	size_t GetAllocationCount() const override;
7592	size_t GetFreeRegionsCount() const override;
7593
7594	void AddDetailedStatistics(VmaDetailedStatistics& inoutStats) const override;
7595	void AddStatistics(VmaStatistics& inoutStats) const override;
7596
7597	#if VMA_STATS_STRING_ENABLED
7598	void PrintDetailedMap(class VmaJsonWriter& json) const override;
7599	#endif
7600
7601	bool CreateAllocationRequest(
7602	VkDeviceSize allocSize,
7603	VkDeviceSize allocAlignment,
7604	bool upperAddress,
7605	VmaSuballocationType allocType,
7606	uint32_t strategy,
7607	VmaAllocationRequest* pAllocationRequest) override;
7608
7609	VkResult CheckCorruption(const void* pBlockData) override;
7610
7611	void Alloc(
7612	const VmaAllocationRequest& request,
7613	VmaSuballocationType type,
7614	void* userData) override;
7615
7616	void Free(VmaAllocHandle allocHandle) override;
7617	void GetAllocationInfo(VmaAllocHandle allocHandle, VmaVirtualAllocationInfo& outInfo) override;
7618	void* GetAllocationUserData(VmaAllocHandle allocHandle) const override;
7619	VmaAllocHandle GetAllocationListBegin() const override;
7620	VmaAllocHandle GetNextAllocation(VmaAllocHandle prevAlloc) const override;
7621	VkDeviceSize GetNextFreeRegionSize(VmaAllocHandle alloc) const override;
7622	void Clear() override;
7623	void SetAllocationUserData(VmaAllocHandle allocHandle, void* userData) override;
7624	void DebugLogAllAllocations() const override;
7625
7626	private:
7627	/*
7628	There are two suballocation vectors, used in ping-pong way.
7629	The one with index m_1stVectorIndex is called 1st.
7630	The one with index (m_1stVectorIndex ^ 1) is called 2nd.
7631	2nd can be non-empty only when 1st is not empty.
7632	When 2nd is not empty, m_2ndVectorMode indicates its mode of operation.
7633	*/
7634	typedef VmaVector<VmaSuballocation, VmaStlAllocator<VmaSuballocation>> SuballocationVectorType;
7635
7636	enum SECOND_VECTOR_MODE
7637	{
7638	SECOND_VECTOR_EMPTY,
7639	/*
7640	Suballocations in 2nd vector are created later than the ones in 1st, but they
7641	all have smaller offset.
7642	*/
7643	SECOND_VECTOR_RING_BUFFER,
7644	/*
7645	Suballocations in 2nd vector are upper side of double stack.
7646	They all have offsets higher than those in 1st vector.
7647	Top of this stack means smaller offsets, but higher indices in this vector.
7648	*/
7649	SECOND_VECTOR_DOUBLE_STACK,
7650	};
7651
7652	VkDeviceSize m_SumFreeSize;
7653	SuballocationVectorType m_Suballocations0, m_Suballocations1;
7654	uint32_t m_1stVectorIndex;
7655	SECOND_VECTOR_MODE m_2ndVectorMode;
7656	// Number of items in 1st vector with hAllocation = null at the beginning.
7657	size_t m_1stNullItemsBeginCount;
7658	// Number of other items in 1st vector with hAllocation = null somewhere in the middle.
7659	size_t m_1stNullItemsMiddleCount;
7660	// Number of items in 2nd vector with hAllocation = null.
7661	size_t m_2ndNullItemsCount;
7662
7663	SuballocationVectorType& AccessSuballocations1st() { return m_1stVectorIndex ? m_Suballocations1 : m_Suballocations0; }
7664	SuballocationVectorType& AccessSuballocations2nd() { return m_1stVectorIndex ? m_Suballocations0 : m_Suballocations1; }
7665	const SuballocationVectorType& AccessSuballocations1st() const { return m_1stVectorIndex ? m_Suballocations1 : m_Suballocations0; }
7666	const SuballocationVectorType& AccessSuballocations2nd() const { return m_1stVectorIndex ? m_Suballocations0 : m_Suballocations1; }
7667
7668	VmaSuballocation& FindSuballocation(VkDeviceSize offset) const;
7669	bool ShouldCompact1st() const;
7670	void CleanupAfterFree();
7671
7672	bool CreateAllocationRequest_LowerAddress(
7673	VkDeviceSize allocSize,
7674	VkDeviceSize allocAlignment,
7675	VmaSuballocationType allocType,
7676	uint32_t strategy,
7677	VmaAllocationRequest* pAllocationRequest);
7678	bool CreateAllocationRequest_UpperAddress(
7679	VkDeviceSize allocSize,
7680	VkDeviceSize allocAlignment,
7681	VmaSuballocationType allocType,
7682	uint32_t strategy,
7683	VmaAllocationRequest* pAllocationRequest);
7684	};
7685
7686	#ifndef _VMA_BLOCK_METADATA_LINEAR_FUNCTIONS
7687	VmaBlockMetadata_Linear::VmaBlockMetadata_Linear(const VkAllocationCallbacks* pAllocationCallbacks,
7688	VkDeviceSize bufferImageGranularity, bool isVirtual)
7689	: VmaBlockMetadata(pAllocationCallbacks, bufferImageGranularity, isVirtual),
7690	m_SumFreeSize(`0`),
7691	m_Suballocations0(VmaStlAllocator<VmaSuballocation>(pAllocationCallbacks)),
7692	m_Suballocations1(VmaStlAllocator<VmaSuballocation>(pAllocationCallbacks)),
7693	m_1stVectorIndex(`0`),
7694	m_2ndVectorMode(SECOND_VECTOR_EMPTY),
7695	m_1stNullItemsBeginCount(`0`),
7696	m_1stNullItemsMiddleCount(`0`),
7697	m_2ndNullItemsCount(`0`) {}
7698
7699	void VmaBlockMetadata_Linear::Init(VkDeviceSize size)
7700	{
7701	VmaBlockMetadata::Init(size);
7702	m_SumFreeSize = size;
7703	}
7704
7705	bool VmaBlockMetadata_Linear::Validate() const
7706	{
7707	const SuballocationVectorType& suballocations1st = AccessSuballocations1st();
7708	const SuballocationVectorType& suballocations2nd = AccessSuballocations2nd();
7709
7710	VMA_VALIDATE(suballocations2nd.empty() == (m_2ndVectorMode == SECOND_VECTOR_EMPTY));
7711	VMA_VALIDATE(!suballocations1st.empty() \|\|
7712	suballocations2nd.empty() \|\|
7713	m_2ndVectorMode != SECOND_VECTOR_RING_BUFFER);
7714
7715	if (!suballocations1st.empty())
7716	{
7717	// Null item at the beginning should be accounted into m_1stNullItemsBeginCount.
7718	VMA_VALIDATE(suballocations1st[m_1stNullItemsBeginCount].type != VMA_SUBALLOCATION_TYPE_FREE);
7719	// Null item at the end should be just pop_back().
7720	VMA_VALIDATE(suballocations1st.back().type != VMA_SUBALLOCATION_TYPE_FREE);
7721	}
7722	if (!suballocations2nd.empty())
7723	{
7724	// Null item at the end should be just pop_back().
7725	VMA_VALIDATE(suballocations2nd.back().type != VMA_SUBALLOCATION_TYPE_FREE);
7726	}
7727
7728	VMA_VALIDATE(m_1stNullItemsBeginCount + m_1stNullItemsMiddleCount <= suballocations1st.size());
7729	VMA_VALIDATE(m_2ndNullItemsCount <= suballocations2nd.size());
7730
7731	VkDeviceSize sumUsedSize = `0`;
7732	const size_t suballoc1stCount = suballocations1st.size();
7733	const VkDeviceSize debugMargin = GetDebugMargin();
7734	VkDeviceSize offset = `0`;
7735
7736	if (m_2ndVectorMode == SECOND_VECTOR_RING_BUFFER)
7737	{
7738	const size_t suballoc2ndCount = suballocations2nd.size();
7739	size_t nullItem2ndCount = `0`;
7740	for (size_t i = `0`; i < suballoc2ndCount; ++i)
7741	{
7742	const VmaSuballocation& suballoc = suballocations2nd[i];
7743	const bool currFree = (suballoc.type == VMA_SUBALLOCATION_TYPE_FREE);
7744
7745	VmaAllocation const alloc = (VmaAllocation)suballoc.userData;
7746	if (!IsVirtual())
7747	{
7748	VMA_VALIDATE(currFree == (alloc == VK_NULL_HANDLE));
7749	}
7750	VMA_VALIDATE(suballoc.offset >= offset);
7751
7752	if (!currFree)
7753	{
7754	if (!IsVirtual())
7755	{
7756	VMA_VALIDATE((VkDeviceSize)alloc->GetAllocHandle() == suballoc.offset + `1`);
7757	VMA_VALIDATE(alloc->GetSize() == suballoc.size);
7758	}
7759	sumUsedSize += suballoc.size;
7760	}
7761	else
7762	{
7763	++nullItem2ndCount;
7764	}
7765
7766	offset = suballoc.offset + suballoc.size + debugMargin;
7767	}
7768
7769	VMA_VALIDATE(nullItem2ndCount == m_2ndNullItemsCount);
7770	}
7771
7772	for (size_t i = `0`; i < m_1stNullItemsBeginCount; ++i)
7773	{
7774	const VmaSuballocation& suballoc = suballocations1st[i];
7775	VMA_VALIDATE(suballoc.type == VMA_SUBALLOCATION_TYPE_FREE &&
7776	suballoc.userData == VMA_NULL);
7777	}
7778
7779	size_t nullItem1stCount = m_1stNullItemsBeginCount;
7780
7781	for (size_t i = m_1stNullItemsBeginCount; i < suballoc1stCount; ++i)
7782	{
7783	const VmaSuballocation& suballoc = suballocations1st[i];
7784	const bool currFree = (suballoc.type == VMA_SUBALLOCATION_TYPE_FREE);
7785
7786	VmaAllocation const alloc = (VmaAllocation)suballoc.userData;
7787	if (!IsVirtual())
7788	{
7789	VMA_VALIDATE(currFree == (alloc == VK_NULL_HANDLE));
7790	}
7791	VMA_VALIDATE(suballoc.offset >= offset);
7792	VMA_VALIDATE(i >= m_1stNullItemsBeginCount \|\| currFree);
7793
7794	if (!currFree)
7795	{
7796	if (!IsVirtual())
7797	{
7798	VMA_VALIDATE((VkDeviceSize)alloc->GetAllocHandle() == suballoc.offset + `1`);
7799	VMA_VALIDATE(alloc->GetSize() == suballoc.size);
7800	}
7801	sumUsedSize += suballoc.size;
7802	}
7803	else
7804	{
7805	++nullItem1stCount;
7806	}
7807
7808	offset = suballoc.offset + suballoc.size + debugMargin;
7809	}
7810	VMA_VALIDATE(nullItem1stCount == m_1stNullItemsBeginCount + m_1stNullItemsMiddleCount);
7811
7812	if (m_2ndVectorMode == SECOND_VECTOR_DOUBLE_STACK)
7813	{
7814	const size_t suballoc2ndCount = suballocations2nd.size();
7815	size_t nullItem2ndCount = `0`;
7816	for (size_t i = suballoc2ndCount; i--; )
7817	{
7818	const VmaSuballocation& suballoc = suballocations2nd[i];
7819	const bool currFree = (suballoc.type == VMA_SUBALLOCATION_TYPE_FREE);
7820
7821	VmaAllocation const alloc = (VmaAllocation)suballoc.userData;
7822	if (!IsVirtual())
7823	{
7824	VMA_VALIDATE(currFree == (alloc == VK_NULL_HANDLE));
7825	}
7826	VMA_VALIDATE(suballoc.offset >= offset);
7827
7828	if (!currFree)
7829	{
7830	if (!IsVirtual())
7831	{
7832	VMA_VALIDATE((VkDeviceSize)alloc->GetAllocHandle() == suballoc.offset + `1`);
7833	VMA_VALIDATE(alloc->GetSize() == suballoc.size);
7834	}
7835	sumUsedSize += suballoc.size;
7836	}
7837	else
7838	{
7839	++nullItem2ndCount;
7840	}
7841
7842	offset = suballoc.offset + suballoc.size + debugMargin;
7843	}
7844
7845	VMA_VALIDATE(nullItem2ndCount == m_2ndNullItemsCount);
7846	}
7847
7848	VMA_VALIDATE(offset <= GetSize());
7849	VMA_VALIDATE(m_SumFreeSize == GetSize() - sumUsedSize);
7850
7851	return true;
7852	}
7853
7854	size_t VmaBlockMetadata_Linear::GetAllocationCount() const
7855	{
7856	return AccessSuballocations1st().size() - m_1stNullItemsBeginCount - m_1stNullItemsMiddleCount +
7857	AccessSuballocations2nd().size() - m_2ndNullItemsCount;
7858	}
7859
7860	size_t VmaBlockMetadata_Linear::GetFreeRegionsCount() const
7861	{
7862	// Function only used for defragmentation, which is disabled for this algorithm
7863	VMA_ASSERT(`0`);
7864	return SIZE_MAX;
7865	}
7866
7867	void VmaBlockMetadata_Linear::AddDetailedStatistics(VmaDetailedStatistics& inoutStats) const
7868	{
7869	const VkDeviceSize size = GetSize();
7870	const SuballocationVectorType& suballocations1st = AccessSuballocations1st();
7871	const SuballocationVectorType& suballocations2nd = AccessSuballocations2nd();
7872	const size_t suballoc1stCount = suballocations1st.size();
7873	const size_t suballoc2ndCount = suballocations2nd.size();
7874
7875	inoutStats.statistics.blockCount++;
7876	inoutStats.statistics.blockBytes += size;
7877
7878	VkDeviceSize lastOffset = `0`;
7879
7880	if (m_2ndVectorMode == SECOND_VECTOR_RING_BUFFER)
7881	{
7882	const VkDeviceSize freeSpace2ndTo1stEnd = suballocations1st[m_1stNullItemsBeginCount].offset;
7883	size_t nextAlloc2ndIndex = `0`;
7884	while (lastOffset < freeSpace2ndTo1stEnd)
7885	{
7886	// Find next non-null allocation or move nextAllocIndex to the end.
7887	while (nextAlloc2ndIndex < suballoc2ndCount &&
7888	suballocations2nd[nextAlloc2ndIndex].userData == VMA_NULL)
7889	{
7890	++nextAlloc2ndIndex;
7891	}
7892
7893	// Found non-null allocation.
7894	if (nextAlloc2ndIndex < suballoc2ndCount)
7895	{
7896	const VmaSuballocation& suballoc = suballocations2nd[nextAlloc2ndIndex];
7897
7898	// 1. Process free space before this allocation.
7899	if (lastOffset < suballoc.offset)
7900	{
7901	// There is free space from lastOffset to suballoc.offset.
7902	const VkDeviceSize unusedRangeSize = suballoc.offset - lastOffset;
7903	VmaAddDetailedStatisticsUnusedRange(inoutStats, unusedRangeSize);
7904	}
7905
7906	// 2. Process this allocation.
7907	// There is allocation with suballoc.offset, suballoc.size.
7908	VmaAddDetailedStatisticsAllocation(inoutStats, suballoc.size);
7909
7910	// 3. Prepare for next iteration.
7911	lastOffset = suballoc.offset + suballoc.size;
7912	++nextAlloc2ndIndex;
7913	}
7914	// We are at the end.
7915	else
7916	{
7917	// There is free space from lastOffset to freeSpace2ndTo1stEnd.
7918	if (lastOffset < freeSpace2ndTo1stEnd)
7919	{
7920	const VkDeviceSize unusedRangeSize = freeSpace2ndTo1stEnd - lastOffset;
7921	VmaAddDetailedStatisticsUnusedRange(inoutStats, unusedRangeSize);
7922	}
7923
7924	// End of loop.
7925	lastOffset = freeSpace2ndTo1stEnd;
7926	}
7927	}
7928	}
7929
7930	size_t nextAlloc1stIndex = m_1stNullItemsBeginCount;
7931	const VkDeviceSize freeSpace1stTo2ndEnd =
7932	m_2ndVectorMode == SECOND_VECTOR_DOUBLE_STACK ? suballocations2nd.back().offset : size;
7933	while (lastOffset < freeSpace1stTo2ndEnd)
7934	{
7935	// Find next non-null allocation or move nextAllocIndex to the end.
7936	while (nextAlloc1stIndex < suballoc1stCount &&
7937	suballocations1st[nextAlloc1stIndex].userData == VMA_NULL)
7938	{
7939	++nextAlloc1stIndex;
7940	}
7941
7942	// Found non-null allocation.
7943	if (nextAlloc1stIndex < suballoc1stCount)
7944	{
7945	const VmaSuballocation& suballoc = suballocations1st[nextAlloc1stIndex];
7946
7947	// 1. Process free space before this allocation.
7948	if (lastOffset < suballoc.offset)
7949	{
7950	// There is free space from lastOffset to suballoc.offset.
7951	const VkDeviceSize unusedRangeSize = suballoc.offset - lastOffset;
7952	VmaAddDetailedStatisticsUnusedRange(inoutStats, unusedRangeSize);
7953	}
7954
7955	// 2. Process this allocation.
7956	// There is allocation with suballoc.offset, suballoc.size.
7957	VmaAddDetailedStatisticsAllocation(inoutStats, suballoc.size);
7958
7959	// 3. Prepare for next iteration.
7960	lastOffset = suballoc.offset + suballoc.size;
7961	++nextAlloc1stIndex;
7962	}
7963	// We are at the end.
7964	else
7965	{
7966	// There is free space from lastOffset to freeSpace1stTo2ndEnd.
7967	if (lastOffset < freeSpace1stTo2ndEnd)
7968	{
7969	const VkDeviceSize unusedRangeSize = freeSpace1stTo2ndEnd - lastOffset;
7970	VmaAddDetailedStatisticsUnusedRange(inoutStats, unusedRangeSize);
7971	}
7972
7973	// End of loop.
7974	lastOffset = freeSpace1stTo2ndEnd;
7975	}
7976	}
7977
7978	if (m_2ndVectorMode == SECOND_VECTOR_DOUBLE_STACK)
7979	{
7980	size_t nextAlloc2ndIndex = suballocations2nd.size() - `1`;
7981	while (lastOffset < size)
7982	{
7983	// Find next non-null allocation or move nextAllocIndex to the end.
7984	while (nextAlloc2ndIndex != SIZE_MAX &&
7985	suballocations2nd[nextAlloc2ndIndex].userData == VMA_NULL)
7986	{
7987	--nextAlloc2ndIndex;
7988	}
7989
7990	// Found non-null allocation.
7991	if (nextAlloc2ndIndex != SIZE_MAX)
7992	{
7993	const VmaSuballocation& suballoc = suballocations2nd[nextAlloc2ndIndex];
7994
7995	// 1. Process free space before this allocation.
7996	if (lastOffset < suballoc.offset)
7997	{
7998	// There is free space from lastOffset to suballoc.offset.
7999	const VkDeviceSize unusedRangeSize = suballoc.offset - lastOffset;
8000	VmaAddDetailedStatisticsUnusedRange(inoutStats, unusedRangeSize);
8001	}
8002
8003	// 2. Process this allocation.
8004	// There is allocation with suballoc.offset, suballoc.size.
8005	VmaAddDetailedStatisticsAllocation(inoutStats, suballoc.size);
8006
8007	// 3. Prepare for next iteration.
8008	lastOffset = suballoc.offset + suballoc.size;
8009	--nextAlloc2ndIndex;
8010	}
8011	// We are at the end.
8012	else
8013	{
8014	// There is free space from lastOffset to size.
8015	if (lastOffset < size)
8016	{
8017	const VkDeviceSize unusedRangeSize = size - lastOffset;
8018	VmaAddDetailedStatisticsUnusedRange(inoutStats, unusedRangeSize);
8019	}
8020
8021	// End of loop.
8022	lastOffset = size;
8023	}
8024	}
8025	}
8026	}
8027
8028	void VmaBlockMetadata_Linear::AddStatistics(VmaStatistics& inoutStats) const
8029	{
8030	const SuballocationVectorType& suballocations1st = AccessSuballocations1st();
8031	const SuballocationVectorType& suballocations2nd = AccessSuballocations2nd();
8032	const VkDeviceSize size = GetSize();
8033	const size_t suballoc1stCount = suballocations1st.size();
8034	const size_t suballoc2ndCount = suballocations2nd.size();
8035
8036	inoutStats.blockCount++;
8037	inoutStats.blockBytes += size;
8038	inoutStats.allocationBytes += size - m_SumFreeSize;
8039
8040	VkDeviceSize lastOffset = `0`;
8041
8042	if (m_2ndVectorMode == SECOND_VECTOR_RING_BUFFER)
8043	{
8044	const VkDeviceSize freeSpace2ndTo1stEnd = suballocations1st[m_1stNullItemsBeginCount].offset;
8045	size_t nextAlloc2ndIndex = m_1stNullItemsBeginCount;
8046	while (lastOffset < freeSpace2ndTo1stEnd)
8047	{
8048	// Find next non-null allocation or move nextAlloc2ndIndex to the end.
8049	while (nextAlloc2ndIndex < suballoc2ndCount &&
8050	suballocations2nd[nextAlloc2ndIndex].userData == VMA_NULL)
8051	{
8052	++nextAlloc2ndIndex;
8053	}
8054
8055	// Found non-null allocation.
8056	if (nextAlloc2ndIndex < suballoc2ndCount)
8057	{
8058	const VmaSuballocation& suballoc = suballocations2nd[nextAlloc2ndIndex];
8059
8060	// 1. Process free space before this allocation.
8061	if (lastOffset < suballoc.offset)
8062	{
8063	// There is free space from lastOffset to suballoc.offset.
8064	const VkDeviceSize unusedRangeSize = suballoc.offset - lastOffset;
8065	}
8066
8067	// 2. Process this allocation.
8068	// There is allocation with suballoc.offset, suballoc.size.
8069	++inoutStats.allocationCount;
8070
8071	// 3. Prepare for next iteration.
8072	lastOffset = suballoc.offset + suballoc.size;
8073	++nextAlloc2ndIndex;
8074	}
8075	// We are at the end.
8076	else
8077	{
8078	if (lastOffset < freeSpace2ndTo1stEnd)
8079	{
8080	// There is free space from lastOffset to freeSpace2ndTo1stEnd.
8081	const VkDeviceSize unusedRangeSize = freeSpace2ndTo1stEnd - lastOffset;
8082	}
8083
8084	// End of loop.
8085	lastOffset = freeSpace2ndTo1stEnd;
8086	}
8087	}
8088	}
8089
8090	size_t nextAlloc1stIndex = m_1stNullItemsBeginCount;
8091	const VkDeviceSize freeSpace1stTo2ndEnd =
8092	m_2ndVectorMode == SECOND_VECTOR_DOUBLE_STACK ? suballocations2nd.back().offset : size;
8093	while (lastOffset < freeSpace1stTo2ndEnd)
8094	{
8095	// Find next non-null allocation or move nextAllocIndex to the end.
8096	while (nextAlloc1stIndex < suballoc1stCount &&
8097	suballocations1st[nextAlloc1stIndex].userData == VMA_NULL)
8098	{
8099	++nextAlloc1stIndex;
8100	}
8101
8102	// Found non-null allocation.
8103	if (nextAlloc1stIndex < suballoc1stCount)
8104	{
8105	const VmaSuballocation& suballoc = suballocations1st[nextAlloc1stIndex];
8106
8107	// 1. Process free space before this allocation.
8108	if (lastOffset < suballoc.offset)
8109	{
8110	// There is free space from lastOffset to suballoc.offset.
8111	const VkDeviceSize unusedRangeSize = suballoc.offset - lastOffset;
8112	}
8113
8114	// 2. Process this allocation.
8115	// There is allocation with suballoc.offset, suballoc.size.
8116	++inoutStats.allocationCount;
8117
8118	// 3. Prepare for next iteration.
8119	lastOffset = suballoc.offset + suballoc.size;
8120	++nextAlloc1stIndex;
8121	}
8122	// We are at the end.
8123	else
8124	{
8125	if (lastOffset < freeSpace1stTo2ndEnd)
8126	{
8127	// There is free space from lastOffset to freeSpace1stTo2ndEnd.
8128	const VkDeviceSize unusedRangeSize = freeSpace1stTo2ndEnd - lastOffset;
8129	}
8130
8131	// End of loop.
8132	lastOffset = freeSpace1stTo2ndEnd;
8133	}
8134	}
8135
8136	if (m_2ndVectorMode == SECOND_VECTOR_DOUBLE_STACK)
8137	{
8138	size_t nextAlloc2ndIndex = suballocations2nd.size() - `1`;
8139	while (lastOffset < size)
8140	{
8141	// Find next non-null allocation or move nextAlloc2ndIndex to the end.
8142	while (nextAlloc2ndIndex != SIZE_MAX &&
8143	suballocations2nd[nextAlloc2ndIndex].userData == VMA_NULL)
8144	{
8145	--nextAlloc2ndIndex;
8146	}
8147
8148	// Found non-null allocation.
8149	if (nextAlloc2ndIndex != SIZE_MAX)
8150	{
8151	const VmaSuballocation& suballoc = suballocations2nd[nextAlloc2ndIndex];
8152
8153	// 1. Process free space before this allocation.
8154	if (lastOffset < suballoc.offset)
8155	{
8156	// There is free space from lastOffset to suballoc.offset.
8157	const VkDeviceSize unusedRangeSize = suballoc.offset - lastOffset;
8158	}
8159
8160	// 2. Process this allocation.
8161	// There is allocation with suballoc.offset, suballoc.size.
8162	++inoutStats.allocationCount;
8163
8164	// 3. Prepare for next iteration.
8165	lastOffset = suballoc.offset + suballoc.size;
8166	--nextAlloc2ndIndex;
8167	}
8168	// We are at the end.
8169	else
8170	{
8171	if (lastOffset < size)
8172	{
8173	// There is free space from lastOffset to size.
8174	const VkDeviceSize unusedRangeSize = size - lastOffset;
8175	}
8176
8177	// End of loop.
8178	lastOffset = size;
8179	}
8180	}
8181	}
8182	}
8183
8184	#if VMA_STATS_STRING_ENABLED
8185	void VmaBlockMetadata_Linear::PrintDetailedMap(class VmaJsonWriter& json) const
8186	{
8187	const VkDeviceSize size = GetSize();
8188	const SuballocationVectorType& suballocations1st = AccessSuballocations1st();
8189	const SuballocationVectorType& suballocations2nd = AccessSuballocations2nd();
8190	const size_t suballoc1stCount = suballocations1st.size();
8191	const size_t suballoc2ndCount = suballocations2nd.size();
8192
8193	// FIRST PASS
8194
8195	size_t unusedRangeCount = `0`;
8196	VkDeviceSize usedBytes = `0`;
8197
8198	VkDeviceSize lastOffset = `0`;
8199
8200	size_t alloc2ndCount = `0`;
8201	if (m_2ndVectorMode == SECOND_VECTOR_RING_BUFFER)
8202	{
8203	const VkDeviceSize freeSpace2ndTo1stEnd = suballocations1st[m_1stNullItemsBeginCount].offset;
8204	size_t nextAlloc2ndIndex = `0`;
8205	while (lastOffset < freeSpace2ndTo1stEnd)
8206	{
8207	// Find next non-null allocation or move nextAlloc2ndIndex to the end.
8208	while (nextAlloc2ndIndex < suballoc2ndCount &&
8209	suballocations2nd[nextAlloc2ndIndex].userData == VMA_NULL)
8210	{
8211	++nextAlloc2ndIndex;
8212	}
8213
8214	// Found non-null allocation.
8215	if (nextAlloc2ndIndex < suballoc2ndCount)
8216	{
8217	const VmaSuballocation& suballoc = suballocations2nd[nextAlloc2ndIndex];
8218
8219	// 1. Process free space before this allocation.
8220	if (lastOffset < suballoc.offset)
8221	{
8222	// There is free space from lastOffset to suballoc.offset.
8223	++unusedRangeCount;
8224	}
8225
8226	// 2. Process this allocation.
8227	// There is allocation with suballoc.offset, suballoc.size.
8228	++alloc2ndCount;
8229	usedBytes += suballoc.size;
8230
8231	// 3. Prepare for next iteration.
8232	lastOffset = suballoc.offset + suballoc.size;
8233	++nextAlloc2ndIndex;
8234	}
8235	// We are at the end.
8236	else
8237	{
8238	if (lastOffset < freeSpace2ndTo1stEnd)
8239	{
8240	// There is free space from lastOffset to freeSpace2ndTo1stEnd.
8241	++unusedRangeCount;
8242	}
8243
8244	// End of loop.
8245	lastOffset = freeSpace2ndTo1stEnd;
8246	}
8247	}
8248	}
8249
8250	size_t nextAlloc1stIndex = m_1stNullItemsBeginCount;
8251	size_t alloc1stCount = `0`;
8252	const VkDeviceSize freeSpace1stTo2ndEnd =
8253	m_2ndVectorMode == SECOND_VECTOR_DOUBLE_STACK ? suballocations2nd.back().offset : size;
8254	while (lastOffset < freeSpace1stTo2ndEnd)
8255	{
8256	// Find next non-null allocation or move nextAllocIndex to the end.
8257	while (nextAlloc1stIndex < suballoc1stCount &&
8258	suballocations1st[nextAlloc1stIndex].userData == VMA_NULL)
8259	{
8260	++nextAlloc1stIndex;
8261	}
8262
8263	// Found non-null allocation.
8264	if (nextAlloc1stIndex < suballoc1stCount)
8265	{
8266	const VmaSuballocation& suballoc = suballocations1st[nextAlloc1stIndex];
8267
8268	// 1. Process free space before this allocation.
8269	if (lastOffset < suballoc.offset)
8270	{
8271	// There is free space from lastOffset to suballoc.offset.
8272	++unusedRangeCount;
8273	}
8274
8275	// 2. Process this allocation.
8276	// There is allocation with suballoc.offset, suballoc.size.
8277	++alloc1stCount;
8278	usedBytes += suballoc.size;
8279
8280	// 3. Prepare for next iteration.
8281	lastOffset = suballoc.offset + suballoc.size;
8282	++nextAlloc1stIndex;
8283	}
8284	// We are at the end.
8285	else
8286	{
8287	if (lastOffset < size)
8288	{
8289	// There is free space from lastOffset to freeSpace1stTo2ndEnd.
8290	++unusedRangeCount;
8291	}
8292
8293	// End of loop.
8294	lastOffset = freeSpace1stTo2ndEnd;
8295	}
8296	}
8297
8298	if (m_2ndVectorMode == SECOND_VECTOR_DOUBLE_STACK)
8299	{
8300	size_t nextAlloc2ndIndex = suballocations2nd.size() - `1`;
8301	while (lastOffset < size)
8302	{
8303	// Find next non-null allocation or move nextAlloc2ndIndex to the end.
8304	while (nextAlloc2ndIndex != SIZE_MAX &&
8305	suballocations2nd[nextAlloc2ndIndex].userData == VMA_NULL)
8306	{
8307	--nextAlloc2ndIndex;
8308	}
8309
8310	// Found non-null allocation.
8311	if (nextAlloc2ndIndex != SIZE_MAX)
8312	{
8313	const VmaSuballocation& suballoc = suballocations2nd[nextAlloc2ndIndex];
8314
8315	// 1. Process free space before this allocation.
8316	if (lastOffset < suballoc.offset)
8317	{
8318	// There is free space from lastOffset to suballoc.offset.
8319	++unusedRangeCount;
8320	}
8321
8322	// 2. Process this allocation.
8323	// There is allocation with suballoc.offset, suballoc.size.
8324	++alloc2ndCount;
8325	usedBytes += suballoc.size;
8326
8327	// 3. Prepare for next iteration.
8328	lastOffset = suballoc.offset + suballoc.size;
8329	--nextAlloc2ndIndex;
8330	}
8331	// We are at the end.
8332	else
8333	{
8334	if (lastOffset < size)
8335	{
8336	// There is free space from lastOffset to size.
8337	++unusedRangeCount;
8338	}
8339
8340	// End of loop.
8341	lastOffset = size;
8342	}
8343	}
8344	}
8345
8346	const VkDeviceSize unusedBytes = size - usedBytes;
8347	PrintDetailedMap_Begin(json, unusedBytes, alloc1stCount + alloc2ndCount, unusedRangeCount);
8348
8349	// SECOND PASS
8350	lastOffset = `0`;
8351
8352	if (m_2ndVectorMode == SECOND_VECTOR_RING_BUFFER)
8353	{
8354	const VkDeviceSize freeSpace2ndTo1stEnd = suballocations1st[m_1stNullItemsBeginCount].offset;
8355	size_t nextAlloc2ndIndex = `0`;
8356	while (lastOffset < freeSpace2ndTo1stEnd)
8357	{
8358	// Find next non-null allocation or move nextAlloc2ndIndex to the end.
8359	while (nextAlloc2ndIndex < suballoc2ndCount &&
8360	suballocations2nd[nextAlloc2ndIndex].userData == VMA_NULL)
8361	{
8362	++nextAlloc2ndIndex;
8363	}
8364
8365	// Found non-null allocation.
8366	if (nextAlloc2ndIndex < suballoc2ndCount)
8367	{
8368	const VmaSuballocation& suballoc = suballocations2nd[nextAlloc2ndIndex];
8369
8370	// 1. Process free space before this allocation.
8371	if (lastOffset < suballoc.offset)
8372	{
8373	// There is free space from lastOffset to suballoc.offset.
8374	const VkDeviceSize unusedRangeSize = suballoc.offset - lastOffset;
8375	PrintDetailedMap_UnusedRange(json, lastOffset, unusedRangeSize);
8376	}
8377
8378	// 2. Process this allocation.
8379	// There is allocation with suballoc.offset, suballoc.size.
8380	PrintDetailedMap_Allocation(json, suballoc.offset, suballoc.size, suballoc.userData);
8381
8382	// 3. Prepare for next iteration.
8383	lastOffset = suballoc.offset + suballoc.size;
8384	++nextAlloc2ndIndex;
8385	}
8386	// We are at the end.
8387	else
8388	{
8389	if (lastOffset < freeSpace2ndTo1stEnd)
8390	{
8391	// There is free space from lastOffset to freeSpace2ndTo1stEnd.
8392	const VkDeviceSize unusedRangeSize = freeSpace2ndTo1stEnd - lastOffset;
8393	PrintDetailedMap_UnusedRange(json, lastOffset, unusedRangeSize);
8394	}
8395
8396	// End of loop.
8397	lastOffset = freeSpace2ndTo1stEnd;
8398	}
8399	}
8400	}
8401
8402	nextAlloc1stIndex = m_1stNullItemsBeginCount;
8403	while (lastOffset < freeSpace1stTo2ndEnd)
8404	{
8405	// Find next non-null allocation or move nextAllocIndex to the end.
8406	while (nextAlloc1stIndex < suballoc1stCount &&
8407	suballocations1st[nextAlloc1stIndex].userData == VMA_NULL)
8408	{
8409	++nextAlloc1stIndex;
8410	}
8411
8412	// Found non-null allocation.
8413	if (nextAlloc1stIndex < suballoc1stCount)
8414	{
8415	const VmaSuballocation& suballoc = suballocations1st[nextAlloc1stIndex];
8416
8417	// 1. Process free space before this allocation.
8418	if (lastOffset < suballoc.offset)
8419	{
8420	// There is free space from lastOffset to suballoc.offset.
8421	const VkDeviceSize unusedRangeSize = suballoc.offset - lastOffset;
8422	PrintDetailedMap_UnusedRange(json, lastOffset, unusedRangeSize);
8423	}
8424
8425	// 2. Process this allocation.
8426	// There is allocation with suballoc.offset, suballoc.size.
8427	PrintDetailedMap_Allocation(json, suballoc.offset, suballoc.size, suballoc.userData);
8428
8429	// 3. Prepare for next iteration.
8430	lastOffset = suballoc.offset + suballoc.size;
8431	++nextAlloc1stIndex;
8432	}
8433	// We are at the end.
8434	else
8435	{
8436	if (lastOffset < freeSpace1stTo2ndEnd)
8437	{
8438	// There is free space from lastOffset to freeSpace1stTo2ndEnd.
8439	const VkDeviceSize unusedRangeSize = freeSpace1stTo2ndEnd - lastOffset;
8440	PrintDetailedMap_UnusedRange(json, lastOffset, unusedRangeSize);
8441	}
8442
8443	// End of loop.
8444	lastOffset = freeSpace1stTo2ndEnd;
8445	}
8446	}
8447
8448	if (m_2ndVectorMode == SECOND_VECTOR_DOUBLE_STACK)
8449	{
8450	size_t nextAlloc2ndIndex = suballocations2nd.size() - `1`;
8451	while (lastOffset < size)
8452	{
8453	// Find next non-null allocation or move nextAlloc2ndIndex to the end.
8454	while (nextAlloc2ndIndex != SIZE_MAX &&
8455	suballocations2nd[nextAlloc2ndIndex].userData == VMA_NULL)
8456	{
8457	--nextAlloc2ndIndex;
8458	}
8459
8460	// Found non-null allocation.
8461	if (nextAlloc2ndIndex != SIZE_MAX)
8462	{
8463	const VmaSuballocation& suballoc = suballocations2nd[nextAlloc2ndIndex];
8464
8465	// 1. Process free space before this allocation.
8466	if (lastOffset < suballoc.offset)
8467	{
8468	// There is free space from lastOffset to suballoc.offset.
8469	const VkDeviceSize unusedRangeSize = suballoc.offset - lastOffset;
8470	PrintDetailedMap_UnusedRange(json, lastOffset, unusedRangeSize);
8471	}
8472
8473	// 2. Process this allocation.
8474	// There is allocation with suballoc.offset, suballoc.size.
8475	PrintDetailedMap_Allocation(json, suballoc.offset, suballoc.size, suballoc.userData);
8476
8477	// 3. Prepare for next iteration.
8478	lastOffset = suballoc.offset + suballoc.size;
8479	--nextAlloc2ndIndex;
8480	}
8481	// We are at the end.
8482	else
8483	{
8484	if (lastOffset < size)
8485	{
8486	// There is free space from lastOffset to size.
8487	const VkDeviceSize unusedRangeSize = size - lastOffset;
8488	PrintDetailedMap_UnusedRange(json, lastOffset, unusedRangeSize);
8489	}
8490
8491	// End of loop.
8492	lastOffset = size;
8493	}
8494	}
8495	}
8496
8497	PrintDetailedMap_End(json);
8498	}
8499	#endif // VMA_STATS_STRING_ENABLED
8500
8501	bool VmaBlockMetadata_Linear::CreateAllocationRequest(
8502	VkDeviceSize allocSize,
8503	VkDeviceSize allocAlignment,
8504	bool upperAddress,
8505	VmaSuballocationType allocType,
8506	uint32_t strategy,
8507	VmaAllocationRequest* pAllocationRequest)
8508	{
8509	VMA_ASSERT(allocSize > `0`);
8510	VMA_ASSERT(allocType != VMA_SUBALLOCATION_TYPE_FREE);
8511	VMA_ASSERT(pAllocationRequest != VMA_NULL);
8512	VMA_HEAVY_ASSERT(Validate());
8513	pAllocationRequest->size = allocSize;
8514	return upperAddress ?
8515	CreateAllocationRequest_UpperAddress(
8516	allocSize, allocAlignment, allocType, strategy, pAllocationRequest) :
8517	CreateAllocationRequest_LowerAddress(
8518	allocSize, allocAlignment, allocType, strategy, pAllocationRequest);
8519	}
8520
8521	VkResult VmaBlockMetadata_Linear::CheckCorruption(const void* pBlockData)
8522	{
8523	VMA_ASSERT(!IsVirtual());
8524	SuballocationVectorType& suballocations1st = AccessSuballocations1st();
8525	for (size_t i = m_1stNullItemsBeginCount, count = suballocations1st.size(); i < count; ++i)
8526	{
8527	const VmaSuballocation& suballoc = suballocations1st[i];
8528	if (suballoc.type != VMA_SUBALLOCATION_TYPE_FREE)
8529	{
8530	if (!VmaValidateMagicValue(pBlockData, suballoc.offset + suballoc.size))
8531	{
8532	VMA_ASSERT(`0` && "MEMORY CORRUPTION DETECTED AFTER VALIDATED ALLOCATION!");
8533	return VK_ERROR_UNKNOWN_COPY;
8534	}
8535	}
8536	}
8537
8538	SuballocationVectorType& suballocations2nd = AccessSuballocations2nd();
8539	for (size_t i = `0`, count = suballocations2nd.size(); i < count; ++i)
8540	{
8541	const VmaSuballocation& suballoc = suballocations2nd[i];
8542	if (suballoc.type != VMA_SUBALLOCATION_TYPE_FREE)
8543	{
8544	if (!VmaValidateMagicValue(pBlockData, suballoc.offset + suballoc.size))
8545	{
8546	VMA_ASSERT(`0` && "MEMORY CORRUPTION DETECTED AFTER VALIDATED ALLOCATION!");
8547	return VK_ERROR_UNKNOWN_COPY;
8548	}
8549	}
8550	}
8551
8552	return VK_SUCCESS;
8553	}
8554
8555	void VmaBlockMetadata_Linear::Alloc(
8556	const VmaAllocationRequest& request,
8557	VmaSuballocationType type,
8558	void* userData)
8559	{
8560	const VkDeviceSize offset = (VkDeviceSize)request.allocHandle - `1`;
8561	const VmaSuballocation newSuballoc = { offset, request.size, userData, type };
8562
8563	switch (request.type)
8564	{
8565	case VmaAllocationRequestType::UpperAddress:
8566	{
8567	VMA_ASSERT(m_2ndVectorMode != SECOND_VECTOR_RING_BUFFER &&
8568	"CRITICAL ERROR: Trying to use linear allocator as double stack while it was already used as ring buffer.");
8569	SuballocationVectorType& suballocations2nd = AccessSuballocations2nd();
8570	suballocations2nd.push_back(newSuballoc);
8571	m_2ndVectorMode = SECOND_VECTOR_DOUBLE_STACK;
8572	}
8573	break;
8574	case VmaAllocationRequestType::EndOf1st:
8575	{
8576	SuballocationVectorType& suballocations1st = AccessSuballocations1st();
8577
8578	VMA_ASSERT(suballocations1st.empty() \|\|
8579	offset >= suballocations1st.back().offset + suballocations1st.back().size);
8580	// Check if it fits before the end of the block.
8581	VMA_ASSERT(offset + request.size <= GetSize());
8582
8583	suballocations1st.push_back(newSuballoc);
8584	}
8585	break;
8586	case VmaAllocationRequestType::EndOf2nd:
8587	{
8588	SuballocationVectorType& suballocations1st = AccessSuballocations1st();
8589	// New allocation at the end of 2-part ring buffer, so before first allocation from 1st vector.
8590	VMA_ASSERT(!suballocations1st.empty() &&
8591	offset + request.size <= suballocations1st[m_1stNullItemsBeginCount].offset);
8592	SuballocationVectorType& suballocations2nd = AccessSuballocations2nd();
8593
8594	switch (m_2ndVectorMode)
8595	{
8596	case SECOND_VECTOR_EMPTY:
8597	// First allocation from second part ring buffer.
8598	VMA_ASSERT(suballocations2nd.empty());
8599	m_2ndVectorMode = SECOND_VECTOR_RING_BUFFER;
8600	break;
8601	case SECOND_VECTOR_RING_BUFFER:
8602	// 2-part ring buffer is already started.
8603	VMA_ASSERT(!suballocations2nd.empty());
8604	break;
8605	case SECOND_VECTOR_DOUBLE_STACK:
8606	VMA_ASSERT(`0` && "CRITICAL ERROR: Trying to use linear allocator as ring buffer while it was already used as double stack.");
8607	break;
8608	default:
8609	VMA_ASSERT(`0`);
8610	}
8611
8612	suballocations2nd.push_back(newSuballoc);
8613	}
8614	break;
8615	default:
8616	VMA_ASSERT(`0` && "CRITICAL INTERNAL ERROR.");
8617	}
8618
8619	m_SumFreeSize -= newSuballoc.size;
8620	}
8621
8622	void VmaBlockMetadata_Linear::Free(VmaAllocHandle allocHandle)
8623	{
8624	SuballocationVectorType& suballocations1st = AccessSuballocations1st();
8625	SuballocationVectorType& suballocations2nd = AccessSuballocations2nd();
8626	VkDeviceSize offset = (VkDeviceSize)allocHandle - `1`;
8627
8628	if (!suballocations1st.empty())
8629	{
8630	// First allocation: Mark it as next empty at the beginning.
8631	VmaSuballocation& firstSuballoc = suballocations1st[m_1stNullItemsBeginCount];
8632	if (firstSuballoc.offset == offset)
8633	{
8634	firstSuballoc.type = VMA_SUBALLOCATION_TYPE_FREE;
8635	firstSuballoc.userData = VMA_NULL;
8636	m_SumFreeSize += firstSuballoc.size;
8637	++m_1stNullItemsBeginCount;
8638	CleanupAfterFree();
8639	return;
8640	}
8641	}
8642
8643	// Last allocation in 2-part ring buffer or top of upper stack (same logic).
8644	if (m_2ndVectorMode == SECOND_VECTOR_RING_BUFFER \|\|
8645	m_2ndVectorMode == SECOND_VECTOR_DOUBLE_STACK)
8646	{
8647	VmaSuballocation& lastSuballoc = suballocations2nd.back();
8648	if (lastSuballoc.offset == offset)
8649	{
8650	m_SumFreeSize += lastSuballoc.size;
8651	suballocations2nd.pop_back();
8652	CleanupAfterFree();
8653	return;
8654	}
8655	}
8656	// Last allocation in 1st vector.
8657	else if (m_2ndVectorMode == SECOND_VECTOR_EMPTY)
8658	{
8659	VmaSuballocation& lastSuballoc = suballocations1st.back();
8660	if (lastSuballoc.offset == offset)
8661	{
8662	m_SumFreeSize += lastSuballoc.size;
8663	suballocations1st.pop_back();
8664	CleanupAfterFree();
8665	return;
8666	}
8667	}
8668
8669	VmaSuballocation refSuballoc;
8670	refSuballoc.offset = offset;
8671	// Rest of members stays uninitialized intentionally for better performance.
8672
8673	// Item from the middle of 1st vector.
8674	{
8675	const SuballocationVectorType::iterator it = VmaBinaryFindSorted(
8676	suballocations1st.begin() + m_1stNullItemsBeginCount,
8677	suballocations1st.end(),
8678	refSuballoc,
8679	VmaSuballocationOffsetLess());
8680	if (it != suballocations1st.end())
8681	{
8682	it->type = VMA_SUBALLOCATION_TYPE_FREE;
8683	it->userData = VMA_NULL;
8684	++m_1stNullItemsMiddleCount;
8685	m_SumFreeSize += it->size;
8686	CleanupAfterFree();
8687	return;
8688	}
8689	}
8690
8691	if (m_2ndVectorMode != SECOND_VECTOR_EMPTY)
8692	{
8693	// Item from the middle of 2nd vector.
8694	const SuballocationVectorType::iterator it = m_2ndVectorMode == SECOND_VECTOR_RING_BUFFER ?
8695	VmaBinaryFindSorted(suballocations2nd.begin(), suballocations2nd.end(), refSuballoc, VmaSuballocationOffsetLess()) :
8696	VmaBinaryFindSorted(suballocations2nd.begin(), suballocations2nd.end(), refSuballoc, VmaSuballocationOffsetGreater());
8697	if (it != suballocations2nd.end())
8698	{
8699	it->type = VMA_SUBALLOCATION_TYPE_FREE;
8700	it->userData = VMA_NULL;
8701	++m_2ndNullItemsCount;
8702	m_SumFreeSize += it->size;
8703	CleanupAfterFree();
8704	return;
8705	}
8706	}
8707
8708	VMA_ASSERT(`0` && "Allocation to free not found in linear allocator!");
8709	}
8710
8711	void VmaBlockMetadata_Linear::GetAllocationInfo(VmaAllocHandle allocHandle, VmaVirtualAllocationInfo& outInfo)
8712	{
8713	outInfo.offset = (VkDeviceSize)allocHandle - `1`;
8714	VmaSuballocation& suballoc = FindSuballocation(outInfo.offset);
8715	outInfo.size = suballoc.size;
8716	outInfo.pUserData = suballoc.userData;
8717	}
8718
8719	void* VmaBlockMetadata_Linear::GetAllocationUserData(VmaAllocHandle allocHandle) const
8720	{
8721	return FindSuballocation((VkDeviceSize)allocHandle - `1`).userData;
8722	}
8723
8724	VmaAllocHandle VmaBlockMetadata_Linear::GetAllocationListBegin() const
8725	{
8726	// Function only used for defragmentation, which is disabled for this algorithm
8727	VMA_ASSERT(`0`);
8728	return VK_NULL_HANDLE;
8729	}
8730
8731	VmaAllocHandle VmaBlockMetadata_Linear::GetNextAllocation(VmaAllocHandle prevAlloc) const
8732	{
8733	// Function only used for defragmentation, which is disabled for this algorithm
8734	VMA_ASSERT(`0`);
8735	return VK_NULL_HANDLE;
8736	}
8737
8738	VkDeviceSize VmaBlockMetadata_Linear::GetNextFreeRegionSize(VmaAllocHandle alloc) const
8739	{
8740	// Function only used for defragmentation, which is disabled for this algorithm
8741	VMA_ASSERT(`0`);
8742	return `0`;
8743	}
8744
8745	void VmaBlockMetadata_Linear::Clear()
8746	{
8747	m_SumFreeSize = GetSize();
8748	m_Suballocations0.clear();
8749	m_Suballocations1.clear();
8750	// Leaving m_1stVectorIndex unchanged - it doesn't matter.
8751	m_2ndVectorMode = SECOND_VECTOR_EMPTY;
8752	m_1stNullItemsBeginCount = `0`;
8753	m_1stNullItemsMiddleCount = `0`;
8754	m_2ndNullItemsCount = `0`;
8755	}
8756
8757	void VmaBlockMetadata_Linear::SetAllocationUserData(VmaAllocHandle allocHandle, void* userData)
8758	{
8759	VmaSuballocation& suballoc = FindSuballocation((VkDeviceSize)allocHandle - `1`);
8760	suballoc.userData = userData;
8761	}
8762
8763	void VmaBlockMetadata_Linear::DebugLogAllAllocations() const
8764	{
8765	const SuballocationVectorType& suballocations1st = AccessSuballocations1st();
8766	for (auto it = suballocations1st.begin() + m_1stNullItemsBeginCount; it != suballocations1st.end(); ++it)
8767	if (it->type != VMA_SUBALLOCATION_TYPE_FREE)
8768	DebugLogAllocation(it->offset, it->size, it->userData);
8769
8770	const SuballocationVectorType& suballocations2nd = AccessSuballocations2nd();
8771	for (auto it = suballocations2nd.begin(); it != suballocations2nd.end(); ++it)
8772	if (it->type != VMA_SUBALLOCATION_TYPE_FREE)
8773	DebugLogAllocation(it->offset, it->size, it->userData);
8774	}
8775
8776	VmaSuballocation& VmaBlockMetadata_Linear::FindSuballocation(VkDeviceSize offset) const
8777	{
8778	const SuballocationVectorType& suballocations1st = AccessSuballocations1st();
8779	const SuballocationVectorType& suballocations2nd = AccessSuballocations2nd();
8780
8781	VmaSuballocation refSuballoc;
8782	refSuballoc.offset = offset;
8783	// Rest of members stays uninitialized intentionally for better performance.
8784
8785	// Item from the 1st vector.
8786	{
8787	SuballocationVectorType::const_iterator it = VmaBinaryFindSorted(
8788	suballocations1st.begin() + m_1stNullItemsBeginCount,
8789	suballocations1st.end(),
8790	refSuballoc,
8791	VmaSuballocationOffsetLess());
8792	if (it != suballocations1st.end())
8793	{
8794	return const_cast<VmaSuballocation&>(*it);
8795	}
8796	}
8797
8798	if (m_2ndVectorMode != SECOND_VECTOR_EMPTY)
8799	{
8800	// Rest of members stays uninitialized intentionally for better performance.
8801	SuballocationVectorType::const_iterator it = m_2ndVectorMode == SECOND_VECTOR_RING_BUFFER ?
8802	VmaBinaryFindSorted(suballocations2nd.begin(), suballocations2nd.end(), refSuballoc, VmaSuballocationOffsetLess()) :
8803	VmaBinaryFindSorted(suballocations2nd.begin(), suballocations2nd.end(), refSuballoc, VmaSuballocationOffsetGreater());
8804	if (it != suballocations2nd.end())
8805	{
8806	return const_cast<VmaSuballocation&>(*it);
8807	}
8808	}
8809
8810	VMA_ASSERT(`0` && "Allocation not found in linear allocator!");
8811	return const_cast<VmaSuballocation&>(suballocations1st.back()); // Should never occur.
8812	}
8813
8814	bool VmaBlockMetadata_Linear::ShouldCompact1st() const
8815	{
8816	const size_t nullItemCount = m_1stNullItemsBeginCount + m_1stNullItemsMiddleCount;
8817	const size_t suballocCount = AccessSuballocations1st().size();
8818	return suballocCount > `32` && nullItemCount * `2` >= (suballocCount - nullItemCount) * `3`;
8819	}
8820
8821	void VmaBlockMetadata_Linear::CleanupAfterFree()
8822	{
8823	SuballocationVectorType& suballocations1st = AccessSuballocations1st();
8824	SuballocationVectorType& suballocations2nd = AccessSuballocations2nd();
8825
8826	if (IsEmpty())
8827	{
8828	suballocations1st.clear();
8829	suballocations2nd.clear();
8830	m_1stNullItemsBeginCount = `0`;
8831	m_1stNullItemsMiddleCount = `0`;
8832	m_2ndNullItemsCount = `0`;
8833	m_2ndVectorMode = SECOND_VECTOR_EMPTY;
8834	}
8835	else
8836	{
8837	const size_t suballoc1stCount = suballocations1st.size();
8838	const size_t nullItem1stCount = m_1stNullItemsBeginCount + m_1stNullItemsMiddleCount;
8839	VMA_ASSERT(nullItem1stCount <= suballoc1stCount);
8840
8841	// Find more null items at the beginning of 1st vector.
8842	while (m_1stNullItemsBeginCount < suballoc1stCount &&
8843	suballocations1st[m_1stNullItemsBeginCount].type == VMA_SUBALLOCATION_TYPE_FREE)
8844	{
8845	++m_1stNullItemsBeginCount;
8846	--m_1stNullItemsMiddleCount;
8847	}
8848
8849	// Find more null items at the end of 1st vector.
8850	while (m_1stNullItemsMiddleCount > `0` &&
8851	suballocations1st.back().type == VMA_SUBALLOCATION_TYPE_FREE)
8852	{
8853	--m_1stNullItemsMiddleCount;
8854	suballocations1st.pop_back();
8855	}
8856
8857	// Find more null items at the end of 2nd vector.
8858	while (m_2ndNullItemsCount > `0` &&
8859	suballocations2nd.back().type == VMA_SUBALLOCATION_TYPE_FREE)
8860	{
8861	--m_2ndNullItemsCount;
8862	suballocations2nd.pop_back();
8863	}
8864
8865	// Find more null items at the beginning of 2nd vector.
8866	while (m_2ndNullItemsCount > `0` &&
8867	suballocations2nd[`0`].type == VMA_SUBALLOCATION_TYPE_FREE)
8868	{
8869	--m_2ndNullItemsCount;
8870	VmaVectorRemove(suballocations2nd, `0`);
8871	}
8872
8873	if (ShouldCompact1st())
8874	{
8875	const size_t nonNullItemCount = suballoc1stCount - nullItem1stCount;
8876	size_t srcIndex = m_1stNullItemsBeginCount;
8877	for (size_t dstIndex = `0`; dstIndex < nonNullItemCount; ++dstIndex)
8878	{
8879	while (suballocations1st[srcIndex].type == VMA_SUBALLOCATION_TYPE_FREE)
8880	{
8881	++srcIndex;
8882	}
8883	if (dstIndex != srcIndex)
8884	{
8885	suballocations1st[dstIndex] = suballocations1st[srcIndex];
8886	}
8887	++srcIndex;
8888	}
8889	suballocations1st.resize(nonNullItemCount);
8890	m_1stNullItemsBeginCount = `0`;
8891	m_1stNullItemsMiddleCount = `0`;
8892	}
8893
8894	// 2nd vector became empty.
8895	if (suballocations2nd.empty())
8896	{
8897	m_2ndVectorMode = SECOND_VECTOR_EMPTY;
8898	}
8899
8900	// 1st vector became empty.
8901	if (suballocations1st.size() - m_1stNullItemsBeginCount == `0`)
8902	{
8903	suballocations1st.clear();
8904	m_1stNullItemsBeginCount = `0`;
8905
8906	if (!suballocations2nd.empty() && m_2ndVectorMode == SECOND_VECTOR_RING_BUFFER)
8907	{
8908	// Swap 1st with 2nd. Now 2nd is empty.
8909	m_2ndVectorMode = SECOND_VECTOR_EMPTY;
8910	m_1stNullItemsMiddleCount = m_2ndNullItemsCount;
8911	while (m_1stNullItemsBeginCount < suballocations2nd.size() &&
8912	suballocations2nd[m_1stNullItemsBeginCount].type == VMA_SUBALLOCATION_TYPE_FREE)
8913	{
8914	++m_1stNullItemsBeginCount;
8915	--m_1stNullItemsMiddleCount;
8916	}
8917	m_2ndNullItemsCount = `0`;
8918	m_1stVectorIndex ^= `1`;
8919	}
8920	}
8921	}
8922
8923	VMA_HEAVY_ASSERT(Validate());
8924	}
8925
8926	bool VmaBlockMetadata_Linear::CreateAllocationRequest_LowerAddress(
8927	VkDeviceSize allocSize,
8928	VkDeviceSize allocAlignment,
8929	VmaSuballocationType allocType,
8930	uint32_t strategy,
8931	VmaAllocationRequest* pAllocationRequest)
8932	{
8933	const VkDeviceSize blockSize = GetSize();
8934	const VkDeviceSize debugMargin = GetDebugMargin();
8935	const VkDeviceSize bufferImageGranularity = GetBufferImageGranularity();
8936	SuballocationVectorType& suballocations1st = AccessSuballocations1st();
8937	SuballocationVectorType& suballocations2nd = AccessSuballocations2nd();
8938
8939	if (m_2ndVectorMode == SECOND_VECTOR_EMPTY \|\| m_2ndVectorMode == SECOND_VECTOR_DOUBLE_STACK)
8940	{
8941	// Try to allocate at the end of 1st vector.
8942
8943	VkDeviceSize resultBaseOffset = `0`;
8944	if (!suballocations1st.empty())
8945	{
8946	const VmaSuballocation& lastSuballoc = suballocations1st.back();
8947	resultBaseOffset = lastSuballoc.offset + lastSuballoc.size + debugMargin;
8948	}
8949
8950	// Start from offset equal to beginning of free space.
8951	VkDeviceSize resultOffset = resultBaseOffset;
8952
8953	// Apply alignment.
8954	resultOffset = VmaAlignUp(resultOffset, allocAlignment);
8955
8956	// Check previous suballocations for BufferImageGranularity conflicts.
8957	// Make bigger alignment if necessary.
8958	if (bufferImageGranularity > `1` && bufferImageGranularity != allocAlignment && !suballocations1st.empty())
8959	{
8960	bool bufferImageGranularityConflict = false;
8961	for (size_t prevSuballocIndex = suballocations1st.size(); prevSuballocIndex--; )
8962	{
8963	const VmaSuballocation& prevSuballoc = suballocations1st[prevSuballocIndex];
8964	if (VmaBlocksOnSamePage(prevSuballoc.offset, prevSuballoc.size, resultOffset, bufferImageGranularity))
8965	{
8966	if (VmaIsBufferImageGranularityConflict(prevSuballoc.type, allocType))
8967	{
8968	bufferImageGranularityConflict = true;
8969	break;
8970	}
8971	}
8972	else
8973	// Already on previous page.
8974	break;
8975	}
8976	if (bufferImageGranularityConflict)
8977	{
8978	resultOffset = VmaAlignUp(resultOffset, bufferImageGranularity);
8979	}
8980	}
8981
8982	const VkDeviceSize freeSpaceEnd = m_2ndVectorMode == SECOND_VECTOR_DOUBLE_STACK ?
8983	suballocations2nd.back().offset : blockSize;
8984
8985	// There is enough free space at the end after alignment.
8986	if (resultOffset + allocSize + debugMargin <= freeSpaceEnd)
8987	{
8988	// Check next suballocations for BufferImageGranularity conflicts.
8989	// If conflict exists, allocation cannot be made here.
8990	if ((allocSize % bufferImageGranularity \|\| resultOffset % bufferImageGranularity) && m_2ndVectorMode == SECOND_VECTOR_DOUBLE_STACK)
8991	{
8992	for (size_t nextSuballocIndex = suballocations2nd.size(); nextSuballocIndex--; )
8993	{
8994	const VmaSuballocation& nextSuballoc = suballocations2nd[nextSuballocIndex];
8995	if (VmaBlocksOnSamePage(resultOffset, allocSize, nextSuballoc.offset, bufferImageGranularity))
8996	{
8997	if (VmaIsBufferImageGranularityConflict(allocType, nextSuballoc.type))
8998	{
8999	return false;
9000	}
9001	}
9002	else
9003	{
9004	// Already on previous page.
9005	break;
9006	}
9007	}
9008	}
9009
9010	// All tests passed: Success.
9011	pAllocationRequest->allocHandle = (VmaAllocHandle)(resultOffset + `1`);
9012	// pAllocationRequest->item, customData unused.
9013	pAllocationRequest->type = VmaAllocationRequestType::EndOf1st;
9014	return true;
9015	}
9016	}
9017
9018	// Wrap-around to end of 2nd vector. Try to allocate there, watching for the
9019	// beginning of 1st vector as the end of free space.
9020	if (m_2ndVectorMode == SECOND_VECTOR_EMPTY \|\| m_2ndVectorMode == SECOND_VECTOR_RING_BUFFER)
9021	{
9022	VMA_ASSERT(!suballocations1st.empty());
9023
9024	VkDeviceSize resultBaseOffset = `0`;
9025	if (!suballocations2nd.empty())
9026	{
9027	const VmaSuballocation& lastSuballoc = suballocations2nd.back();
9028	resultBaseOffset = lastSuballoc.offset + lastSuballoc.size + debugMargin;
9029	}
9030
9031	// Start from offset equal to beginning of free space.
9032	VkDeviceSize resultOffset = resultBaseOffset;
9033
9034	// Apply alignment.
9035	resultOffset = VmaAlignUp(resultOffset, allocAlignment);
9036
9037	// Check previous suballocations for BufferImageGranularity conflicts.
9038	// Make bigger alignment if necessary.
9039	if (bufferImageGranularity > `1` && bufferImageGranularity != allocAlignment && !suballocations2nd.empty())
9040	{
9041	bool bufferImageGranularityConflict = false;
9042	for (size_t prevSuballocIndex = suballocations2nd.size(); prevSuballocIndex--; )
9043	{
9044	const VmaSuballocation& prevSuballoc = suballocations2nd[prevSuballocIndex];
9045	if (VmaBlocksOnSamePage(prevSuballoc.offset, prevSuballoc.size, resultOffset, bufferImageGranularity))
9046	{
9047	if (VmaIsBufferImageGranularityConflict(prevSuballoc.type, allocType))
9048	{
9049	bufferImageGranularityConflict = true;
9050	break;
9051	}
9052	}
9053	else
9054	// Already on previous page.
9055	break;
9056	}
9057	if (bufferImageGranularityConflict)
9058	{
9059	resultOffset = VmaAlignUp(resultOffset, bufferImageGranularity);
9060	}
9061	}
9062
9063	size_t index1st = m_1stNullItemsBeginCount;
9064
9065	// There is enough free space at the end after alignment.
9066	if ((index1st == suballocations1st.size() && resultOffset + allocSize + debugMargin <= blockSize) \|\|
9067	(index1st < suballocations1st.size() && resultOffset + allocSize + debugMargin <= suballocations1st[index1st].offset))
9068	{
9069	// Check next suballocations for BufferImageGranularity conflicts.
9070	// If conflict exists, allocation cannot be made here.
9071	if (allocSize % bufferImageGranularity \|\| resultOffset % bufferImageGranularity)
9072	{
9073	for (size_t nextSuballocIndex = index1st;
9074	nextSuballocIndex < suballocations1st.size();
9075	nextSuballocIndex++)
9076	{
9077	const VmaSuballocation& nextSuballoc = suballocations1st[nextSuballocIndex];
9078	if (VmaBlocksOnSamePage(resultOffset, allocSize, nextSuballoc.offset, bufferImageGranularity))
9079	{
9080	if (VmaIsBufferImageGranularityConflict(allocType, nextSuballoc.type))
9081	{
9082	return false;
9083	}
9084	}
9085	else
9086	{
9087	// Already on next page.
9088	break;
9089	}
9090	}
9091	}
9092
9093	// All tests passed: Success.
9094	pAllocationRequest->allocHandle = (VmaAllocHandle)(resultOffset + `1`);
9095	pAllocationRequest->type = VmaAllocationRequestType::EndOf2nd;
9096	// pAllocationRequest->item, customData unused.
9097	return true;
9098	}
9099	}
9100
9101	return false;
9102	}
9103
9104	bool VmaBlockMetadata_Linear::CreateAllocationRequest_UpperAddress(
9105	VkDeviceSize allocSize,
9106	VkDeviceSize allocAlignment,
9107	VmaSuballocationType allocType,
9108	uint32_t strategy,
9109	VmaAllocationRequest* pAllocationRequest)
9110	{
9111	const VkDeviceSize blockSize = GetSize();
9112	const VkDeviceSize bufferImageGranularity = GetBufferImageGranularity();
9113	SuballocationVectorType& suballocations1st = AccessSuballocations1st();
9114	SuballocationVectorType& suballocations2nd = AccessSuballocations2nd();
9115
9116	if (m_2ndVectorMode == SECOND_VECTOR_RING_BUFFER)
9117	{
9118	VMA_ASSERT(`0` && "Trying to use pool with linear algorithm as double stack, while it is already being used as ring buffer.");
9119	return false;
9120	}
9121
9122	// Try to allocate before 2nd.back(), or end of block if 2nd.empty().
9123	if (allocSize > blockSize)
9124	{
9125	return false;
9126	}
9127	VkDeviceSize resultBaseOffset = blockSize - allocSize;
9128	if (!suballocations2nd.empty())
9129	{
9130	const VmaSuballocation& lastSuballoc = suballocations2nd.back();
9131	resultBaseOffset = lastSuballoc.offset - allocSize;
9132	if (allocSize > lastSuballoc.offset)
9133	{
9134	return false;
9135	}
9136	}
9137
9138	// Start from offset equal to end of free space.
9139	VkDeviceSize resultOffset = resultBaseOffset;
9140
9141	const VkDeviceSize debugMargin = GetDebugMargin();
9142
9143	// Apply debugMargin at the end.
9144	if (debugMargin > `0`)
9145	{
9146	if (resultOffset < debugMargin)
9147	{
9148	return false;
9149	}
9150	resultOffset -= debugMargin;
9151	}
9152
9153	// Apply alignment.
9154	resultOffset = VmaAlignDown(resultOffset, allocAlignment);
9155
9156	// Check next suballocations from 2nd for BufferImageGranularity conflicts.
9157	// Make bigger alignment if necessary.
9158	if (bufferImageGranularity > `1` && bufferImageGranularity != allocAlignment && !suballocations2nd.empty())
9159	{
9160	bool bufferImageGranularityConflict = false;
9161	for (size_t nextSuballocIndex = suballocations2nd.size(); nextSuballocIndex--; )
9162	{
9163	const VmaSuballocation& nextSuballoc = suballocations2nd[nextSuballocIndex];
9164	if (VmaBlocksOnSamePage(resultOffset, allocSize, nextSuballoc.offset, bufferImageGranularity))
9165	{
9166	if (VmaIsBufferImageGranularityConflict(nextSuballoc.type, allocType))
9167	{
9168	bufferImageGranularityConflict = true;
9169	break;
9170	}
9171	}
9172	else
9173	// Already on previous page.
9174	break;
9175	}
9176	if (bufferImageGranularityConflict)
9177	{
9178	resultOffset = VmaAlignDown(resultOffset, bufferImageGranularity);
9179	}
9180	}
9181
9182	// There is enough free space.
9183	const VkDeviceSize endOf1st = !suballocations1st.empty() ?
9184	suballocations1st.back().offset + suballocations1st.back().size :
9185	`0`;
9186	if (endOf1st + debugMargin <= resultOffset)
9187	{
9188	// Check previous suballocations for BufferImageGranularity conflicts.
9189	// If conflict exists, allocation cannot be made here.
9190	if (bufferImageGranularity > `1`)
9191	{
9192	for (size_t prevSuballocIndex = suballocations1st.size(); prevSuballocIndex--; )
9193	{
9194	const VmaSuballocation& prevSuballoc = suballocations1st[prevSuballocIndex];
9195	if (VmaBlocksOnSamePage(prevSuballoc.offset, prevSuballoc.size, resultOffset, bufferImageGranularity))
9196	{
9197	if (VmaIsBufferImageGranularityConflict(allocType, prevSuballoc.type))
9198	{
9199	return false;
9200	}
9201	}
9202	else
9203	{
9204	// Already on next page.
9205	break;
9206	}
9207	}
9208	}
9209
9210	// All tests passed: Success.
9211	pAllocationRequest->allocHandle = (VmaAllocHandle)(resultOffset + `1`);
9212	// pAllocationRequest->item unused.
9213	pAllocationRequest->type = VmaAllocationRequestType::UpperAddress;
9214	return true;
9215	}
9216
9217	return false;
9218	}
9219	#endif // _VMA_BLOCK_METADATA_LINEAR_FUNCTIONS
9220	#endif // _VMA_BLOCK_METADATA_LINEAR
9221
9222	#if 0
9223	#ifndef _VMA_BLOCK_METADATA_BUDDY
9224	/*
9225	- GetSize() is the original size of allocated memory block.
9226	- m_UsableSize is this size aligned down to a power of two.
9227	All allocations and calculations happen relative to m_UsableSize.
9228	- GetUnusableSize() is the difference between them.
9229	It is reported as separate, unused range, not available for allocations.
9230
9231	Node at level 0 has size = m_UsableSize.
9232	Each next level contains nodes with size 2 times smaller than current level.
9233	m_LevelCount is the maximum number of levels to use in the current object.
9234	*/
9235	class VmaBlockMetadata_Buddy : public VmaBlockMetadata
9236	{
9237	VMA_CLASS_NO_COPY(VmaBlockMetadata_Buddy)
9238	public:
9239	VmaBlockMetadata_Buddy(const VkAllocationCallbacks* pAllocationCallbacks,
9240	VkDeviceSize bufferImageGranularity, bool isVirtual);
9241	virtual ~VmaBlockMetadata_Buddy();
9242
9243	size_t GetAllocationCount() const override { return m_AllocationCount; }
9244	VkDeviceSize GetSumFreeSize() const override { return m_SumFreeSize + GetUnusableSize(); }
9245	bool IsEmpty() const override { return m_Root->type == Node::TYPE_FREE; }
9246	VkResult CheckCorruption(const void* pBlockData) override { return VK_ERROR_FEATURE_NOT_PRESENT; }
9247	VkDeviceSize GetAllocationOffset(VmaAllocHandle allocHandle) const override { return (VkDeviceSize)allocHandle - `1`; };
9248	void DebugLogAllAllocations() const override { DebugLogAllAllocationNode(m_Root, `0`); }
9249
9250	void Init(VkDeviceSize size) override;
9251	bool Validate() const override;
9252
9253	void AddDetailedStatistics(VmaDetailedStatistics& inoutStats) const override;
9254	void AddStatistics(VmaStatistics& inoutStats) const override;
9255
9256	#if VMA_STATS_STRING_ENABLED
9257	void PrintDetailedMap(class VmaJsonWriter& json, uint32_t mapRefCount) const override;
9258	#endif
9259
9260	bool CreateAllocationRequest(
9261	VkDeviceSize allocSize,
9262	VkDeviceSize allocAlignment,
9263	bool upperAddress,
9264	VmaSuballocationType allocType,
9265	uint32_t strategy,
9266	VmaAllocationRequest* pAllocationRequest) override;
9267
9268	void Alloc(
9269	const VmaAllocationRequest& request,
9270	VmaSuballocationType type,
9271	void* userData) override;
9272
9273	void Free(VmaAllocHandle allocHandle) override;
9274	void GetAllocationInfo(VmaAllocHandle allocHandle, VmaVirtualAllocationInfo& outInfo) override;
9275	void* GetAllocationUserData(VmaAllocHandle allocHandle) const override;
9276	VmaAllocHandle GetAllocationListBegin() const override;
9277	VmaAllocHandle GetNextAllocation(VmaAllocHandle prevAlloc) const override;
9278	void Clear() override;
9279	void SetAllocationUserData(VmaAllocHandle allocHandle, void* userData) override;
9280
9281	private:
9282	static const size_t MAX_LEVELS = `48`;
9283
9284	struct ValidationContext
9285	{
9286	size_t calculatedAllocationCount = `0`;
9287	size_t calculatedFreeCount = `0`;
9288	VkDeviceSize calculatedSumFreeSize = `0`;
9289	};
9290	struct Node
9291	{
9292	VkDeviceSize offset;
9293	enum TYPE
9294	{
9295	TYPE_FREE,
9296	TYPE_ALLOCATION,
9297	TYPE_SPLIT,
9298	TYPE_COUNT
9299	} type;
9300	Node* parent;
9301	Node* buddy;
9302
9303	union
9304	{
9305	struct
9306	{
9307	Node* prev;
9308	Node* next;
9309	} free;
9310	struct
9311	{
9312	void* userData;
9313	} allocation;
9314	struct
9315	{
9316	Node* leftChild;
9317	} split;
9318	};
9319	};
9320
9321	// Size of the memory block aligned down to a power of two.
9322	VkDeviceSize m_UsableSize;
9323	uint32_t m_LevelCount;
9324	VmaPoolAllocator<Node> m_NodeAllocator;
9325	Node* m_Root;
9326	struct
9327	{
9328	Node* front;
9329	Node* back;
9330	} m_FreeList[MAX_LEVELS];
9331
9332	// Number of nodes in the tree with type == TYPE_ALLOCATION.
9333	size_t m_AllocationCount;
9334	// Number of nodes in the tree with type == TYPE_FREE.
9335	size_t m_FreeCount;
9336	// Doesn't include space wasted due to internal fragmentation - allocation sizes are just aligned up to node sizes.
9337	// Doesn't include unusable size.
9338	VkDeviceSize m_SumFreeSize;
9339
9340	VkDeviceSize GetUnusableSize() const { return GetSize() - m_UsableSize; }
9341	VkDeviceSize LevelToNodeSize(uint32_t level) const { return m_UsableSize >> level; }
9342
9343	VkDeviceSize AlignAllocationSize(VkDeviceSize size) const
9344	{
9345	if (!IsVirtual())
9346	{
9347	size = VmaAlignUp(size, (VkDeviceSize)`16`);
9348	}
9349	return VmaNextPow2(size);
9350	}
9351	Node* FindAllocationNode(VkDeviceSize offset, uint32_t& outLevel) const;
9352	void DeleteNodeChildren(Node* node);
9353	bool ValidateNode(ValidationContext& ctx, const Node* parent, const Node* curr, uint32_t level, VkDeviceSize levelNodeSize) const;
9354	uint32_t AllocSizeToLevel(VkDeviceSize allocSize) const;
9355	void AddNodeToDetailedStatistics(VmaDetailedStatistics& inoutStats, const Node* node, VkDeviceSize levelNodeSize) const;
9356	// Adds node to the front of FreeList at given level.
9357	// node->type must be FREE.
9358	// node->free.prev, next can be undefined.
9359	void AddToFreeListFront(uint32_t level, Node* node);
9360	// Removes node from FreeList at given level.
9361	// node->type must be FREE.
9362	// node->free.prev, next stay untouched.
9363	void RemoveFromFreeList(uint32_t level, Node* node);
9364	void DebugLogAllAllocationNode(Node* node, uint32_t level) const;
9365
9366	#if VMA_STATS_STRING_ENABLED
9367	void PrintDetailedMapNode(class VmaJsonWriter& json, const Node* node, VkDeviceSize levelNodeSize) const;
9368	#endif
9369	};
9370
9371	#ifndef _VMA_BLOCK_METADATA_BUDDY_FUNCTIONS
9372	VmaBlockMetadata_Buddy::VmaBlockMetadata_Buddy(const VkAllocationCallbacks* pAllocationCallbacks,
9373	VkDeviceSize bufferImageGranularity, bool isVirtual)
9374	: VmaBlockMetadata(pAllocationCallbacks, bufferImageGranularity, isVirtual),
9375	m_NodeAllocator(pAllocationCallbacks, `32`), // firstBlockCapacity
9376	m_Root(VMA_NULL),
9377	m_AllocationCount(`0`),
9378	m_FreeCount(`1`),
9379	m_SumFreeSize(`0`)
9380	{
9381	memset(m_FreeList, `0`, sizeof(m_FreeList));
9382	}
9383
9384	VmaBlockMetadata_Buddy::~VmaBlockMetadata_Buddy()
9385	{
9386	DeleteNodeChildren(m_Root);
9387	m_NodeAllocator.Free(m_Root);
9388	}
9389
9390	void VmaBlockMetadata_Buddy::Init(VkDeviceSize size)
9391	{
9392	VmaBlockMetadata::Init(size);
9393
9394	m_UsableSize = VmaPrevPow2(size);
9395	m_SumFreeSize = m_UsableSize;
9396
9397	// Calculate m_LevelCount.
9398	const VkDeviceSize minNodeSize = IsVirtual() ? `1` : `16`;
9399	m_LevelCount = `1`;
9400	while (m_LevelCount < MAX_LEVELS &&
9401	LevelToNodeSize(m_LevelCount) >= minNodeSize)
9402	{
9403	++m_LevelCount;
9404	}
9405
9406	Node* rootNode = m_NodeAllocator.Alloc();
9407	rootNode->offset = `0`;
9408	rootNode->type = Node::TYPE_FREE;
9409	rootNode->parent = VMA_NULL;
9410	rootNode->buddy = VMA_NULL;
9411
9412	m_Root = rootNode;
9413	AddToFreeListFront(`0`, rootNode);
9414	}
9415
9416	bool VmaBlockMetadata_Buddy::Validate() const
9417	{
9418	// Validate tree.
9419	ValidationContext ctx;
9420	if (!ValidateNode(ctx, VMA_NULL, m_Root, `0`, LevelToNodeSize(`0`)))
9421	{
9422	VMA_VALIDATE(false && "ValidateNode failed.");
9423	}
9424	VMA_VALIDATE(m_AllocationCount == ctx.calculatedAllocationCount);
9425	VMA_VALIDATE(m_SumFreeSize == ctx.calculatedSumFreeSize);
9426
9427	// Validate free node lists.
9428	for (uint32_t level = `0`; level < m_LevelCount; ++level)
9429	{
9430	VMA_VALIDATE(m_FreeList[level].front == VMA_NULL \|\|
9431	m_FreeList[level].front->free.prev == VMA_NULL);
9432
9433	for (Node* node = m_FreeList[level].front;
9434	node != VMA_NULL;
9435	node = node->free.next)
9436	{
9437	VMA_VALIDATE(node->type == Node::TYPE_FREE);
9438
9439	if (node->free.next == VMA_NULL)
9440	{
9441	VMA_VALIDATE(m_FreeList[level].back == node);
9442	}
9443	else
9444	{
9445	VMA_VALIDATE(node->free.next->free.prev == node);
9446	}
9447	}
9448	}
9449
9450	// Validate that free lists ar higher levels are empty.
9451	for (uint32_t level = m_LevelCount; level < MAX_LEVELS; ++level)
9452	{
9453	VMA_VALIDATE(m_FreeList[level].front == VMA_NULL && m_FreeList[level].back == VMA_NULL);
9454	}
9455
9456	return true;
9457	}
9458
9459	void VmaBlockMetadata_Buddy::AddDetailedStatistics(VmaDetailedStatistics& inoutStats) const
9460	{
9461	inoutStats.statistics.blockCount++;
9462	inoutStats.statistics.blockBytes += GetSize();
9463
9464	AddNodeToDetailedStatistics(inoutStats, m_Root, LevelToNodeSize(`0`));
9465
9466	const VkDeviceSize unusableSize = GetUnusableSize();
9467	if (unusableSize > `0`)
9468	VmaAddDetailedStatisticsUnusedRange(inoutStats, unusableSize);
9469	}
9470
9471	void VmaBlockMetadata_Buddy::AddStatistics(VmaStatistics& inoutStats) const
9472	{
9473	inoutStats.blockCount++;
9474	inoutStats.allocationCount += (uint32_t)m_AllocationCount;
9475	inoutStats.blockBytes += GetSize();
9476	inoutStats.allocationBytes += GetSize() - m_SumFreeSize;
9477	}
9478
9479	#if VMA_STATS_STRING_ENABLED
9480	void VmaBlockMetadata_Buddy::PrintDetailedMap(class VmaJsonWriter& json, uint32_t mapRefCount) const
9481	{
9482	VmaDetailedStatistics stats;
9483	VmaClearDetailedStatistics(stats);
9484	AddDetailedStatistics(stats);
9485
9486	PrintDetailedMap_Begin(
9487	json,
9488	stats.statistics.blockBytes - stats.statistics.allocationBytes,
9489	stats.statistics.allocationCount,
9490	stats.unusedRangeCount,
9491	mapRefCount);
9492
9493	PrintDetailedMapNode(json, m_Root, LevelToNodeSize(`0`));
9494
9495	const VkDeviceSize unusableSize = GetUnusableSize();
9496	if (unusableSize > `0`)
9497	{
9498	PrintDetailedMap_UnusedRange(json,
9499	m_UsableSize, // offset
9500	unusableSize); // size
9501	}
9502
9503	PrintDetailedMap_End(json);
9504	}
9505	#endif // VMA_STATS_STRING_ENABLED
9506
9507	bool VmaBlockMetadata_Buddy::CreateAllocationRequest(
9508	VkDeviceSize allocSize,
9509	VkDeviceSize allocAlignment,
9510	bool upperAddress,
9511	VmaSuballocationType allocType,
9512	uint32_t strategy,
9513	VmaAllocationRequest* pAllocationRequest)
9514	{
9515	VMA_ASSERT(!upperAddress && "VMA_ALLOCATION_CREATE_UPPER_ADDRESS_BIT can be used only with linear algorithm.");
9516
9517	allocSize = AlignAllocationSize(allocSize);
9518
9519	// Simple way to respect bufferImageGranularity. May be optimized some day.
9520	// Whenever it might be an OPTIMAL image...
9521	if (allocType == VMA_SUBALLOCATION_TYPE_UNKNOWN \|\|
9522	allocType == VMA_SUBALLOCATION_TYPE_IMAGE_UNKNOWN \|\|
9523	allocType == VMA_SUBALLOCATION_TYPE_IMAGE_OPTIMAL)
9524	{
9525	allocAlignment = VMA_MAX(allocAlignment, GetBufferImageGranularity());
9526	allocSize = VmaAlignUp(allocSize, GetBufferImageGranularity());
9527	}
9528
9529	if (allocSize > m_UsableSize)
9530	{
9531	return false;
9532	}
9533
9534	const uint32_t targetLevel = AllocSizeToLevel(allocSize);
9535	for (uint32_t level = targetLevel; level--; )
9536	{
9537	for (Node* freeNode = m_FreeList[level].front;
9538	freeNode != VMA_NULL;
9539	freeNode = freeNode->free.next)
9540	{
9541	if (freeNode->offset % allocAlignment == `0`)
9542	{
9543	pAllocationRequest->type = VmaAllocationRequestType::Normal;
9544	pAllocationRequest->allocHandle = (VmaAllocHandle)(freeNode->offset + `1`);
9545	pAllocationRequest->size = allocSize;
9546	pAllocationRequest->customData = (void*)(uintptr_t)level;
9547	return true;
9548	}
9549	}
9550	}
9551
9552	return false;
9553	}
9554
9555	void VmaBlockMetadata_Buddy::Alloc(
9556	const VmaAllocationRequest& request,
9557	VmaSuballocationType type,
9558	void* userData)
9559	{
9560	VMA_ASSERT(request.type == VmaAllocationRequestType::Normal);
9561
9562	const uint32_t targetLevel = AllocSizeToLevel(request.size);
9563	uint32_t currLevel = (uint32_t)(uintptr_t)request.customData;
9564
9565	Node* currNode = m_FreeList[currLevel].front;
9566	VMA_ASSERT(currNode != VMA_NULL && currNode->type == Node::TYPE_FREE);
9567	const VkDeviceSize offset = (VkDeviceSize)request.allocHandle - `1`;
9568	while (currNode->offset != offset)
9569	{
9570	currNode = currNode->free.next;
9571	VMA_ASSERT(currNode != VMA_NULL && currNode->type == Node::TYPE_FREE);
9572	}
9573
9574	// Go down, splitting free nodes.
9575	while (currLevel < targetLevel)
9576	{
9577	// currNode is already first free node at currLevel.
9578	// Remove it from list of free nodes at this currLevel.
9579	RemoveFromFreeList(currLevel, currNode);
9580
9581	const uint32_t childrenLevel = currLevel + `1`;
9582
9583	// Create two free sub-nodes.
9584	Node* leftChild = m_NodeAllocator.Alloc();
9585	Node* rightChild = m_NodeAllocator.Alloc();
9586
9587	leftChild->offset = currNode->offset;
9588	leftChild->type = Node::TYPE_FREE;
9589	leftChild->parent = currNode;
9590	leftChild->buddy = rightChild;
9591
9592	rightChild->offset = currNode->offset + LevelToNodeSize(childrenLevel);
9593	rightChild->type = Node::TYPE_FREE;
9594	rightChild->parent = currNode;
9595	rightChild->buddy = leftChild;
9596
9597	// Convert current currNode to split type.
9598	currNode->type = Node::TYPE_SPLIT;
9599	currNode->split.leftChild = leftChild;
9600
9601	// Add child nodes to free list. Order is important!
9602	AddToFreeListFront(childrenLevel, rightChild);
9603	AddToFreeListFront(childrenLevel, leftChild);
9604
9605	++m_FreeCount;
9606	++currLevel;
9607	currNode = m_FreeList[currLevel].front;
9608
9609	/*
9610	We can be sure that currNode, as left child of node previously split,
9611	also fulfills the alignment requirement.
9612	*/
9613	}
9614
9615	// Remove from free list.
9616	VMA_ASSERT(currLevel == targetLevel &&
9617	currNode != VMA_NULL &&
9618	currNode->type == Node::TYPE_FREE);
9619	RemoveFromFreeList(currLevel, currNode);
9620
9621	// Convert to allocation node.
9622	currNode->type = Node::TYPE_ALLOCATION;
9623	currNode->allocation.userData = userData;
9624
9625	++m_AllocationCount;
9626	--m_FreeCount;
9627	m_SumFreeSize -= request.size;
9628	}
9629
9630	void VmaBlockMetadata_Buddy::GetAllocationInfo(VmaAllocHandle allocHandle, VmaVirtualAllocationInfo& outInfo)
9631	{
9632	uint32_t level = `0`;
9633	outInfo.offset = (VkDeviceSize)allocHandle - `1`;
9634	const Node* const node = FindAllocationNode(outInfo.offset, level);
9635	outInfo.size = LevelToNodeSize(level);
9636	outInfo.pUserData = node->allocation.userData;
9637	}
9638
9639	void* VmaBlockMetadata_Buddy::GetAllocationUserData(VmaAllocHandle allocHandle) const
9640	{
9641	uint32_t level = `0`;
9642	const Node* const node = FindAllocationNode((VkDeviceSize)allocHandle - `1`, level);
9643	return node->allocation.userData;
9644	}
9645
9646	VmaAllocHandle VmaBlockMetadata_Buddy::GetAllocationListBegin() const
9647	{
9648	// Function only used for defragmentation, which is disabled for this algorithm
9649	return VK_NULL_HANDLE;
9650	}
9651
9652	VmaAllocHandle VmaBlockMetadata_Buddy::GetNextAllocation(VmaAllocHandle prevAlloc) const
9653	{
9654	// Function only used for defragmentation, which is disabled for this algorithm
9655	return VK_NULL_HANDLE;
9656	}
9657
9658	void VmaBlockMetadata_Buddy::DeleteNodeChildren(Node* node)
9659	{
9660	if (node->type == Node::TYPE_SPLIT)
9661	{
9662	DeleteNodeChildren(node->split.leftChild->buddy);
9663	DeleteNodeChildren(node->split.leftChild);
9664	const VkAllocationCallbacks* allocationCallbacks = GetAllocationCallbacks();
9665	m_NodeAllocator.Free(node->split.leftChild->buddy);
9666	m_NodeAllocator.Free(node->split.leftChild);
9667	}
9668	}
9669
9670	void VmaBlockMetadata_Buddy::Clear()
9671	{
9672	DeleteNodeChildren(m_Root);
9673	m_Root->type = Node::TYPE_FREE;
9674	m_AllocationCount = `0`;
9675	m_FreeCount = `1`;
9676	m_SumFreeSize = m_UsableSize;
9677	}
9678
9679	void VmaBlockMetadata_Buddy::SetAllocationUserData(VmaAllocHandle allocHandle, void* userData)
9680	{
9681	uint32_t level = `0`;
9682	Node* const node = FindAllocationNode((VkDeviceSize)allocHandle - `1`, level);
9683	node->allocation.userData = userData;
9684	}
9685
9686	VmaBlockMetadata_Buddy::Node* VmaBlockMetadata_Buddy::FindAllocationNode(VkDeviceSize offset, uint32_t& outLevel) const
9687	{
9688	Node* node = m_Root;
9689	VkDeviceSize nodeOffset = `0`;
9690	outLevel = `0`;
9691	VkDeviceSize levelNodeSize = LevelToNodeSize(`0`);
9692	while (node->type == Node::TYPE_SPLIT)
9693	{
9694	const VkDeviceSize nextLevelNodeSize = levelNodeSize >> `1`;
9695	if (offset < nodeOffset + nextLevelNodeSize)
9696	{
9697	node = node->split.leftChild;
9698	}
9699	else
9700	{
9701	node = node->split.leftChild->buddy;
9702	nodeOffset += nextLevelNodeSize;
9703	}
9704	++outLevel;
9705	levelNodeSize = nextLevelNodeSize;
9706	}
9707
9708	VMA_ASSERT(node != VMA_NULL && node->type == Node::TYPE_ALLOCATION);
9709	return node;
9710	}
9711
9712	bool VmaBlockMetadata_Buddy::ValidateNode(ValidationContext& ctx, const Node* parent, const Node* curr, uint32_t level, VkDeviceSize levelNodeSize) const
9713	{
9714	VMA_VALIDATE(level < m_LevelCount);
9715	VMA_VALIDATE(curr->parent == parent);
9716	VMA_VALIDATE((curr->buddy == VMA_NULL) == (parent == VMA_NULL));
9717	VMA_VALIDATE(curr->buddy == VMA_NULL \|\| curr->buddy->buddy == curr);
9718	switch (curr->type)
9719	{
9720	case Node::TYPE_FREE:
9721	// curr->free.prev, next are validated separately.
9722	ctx.calculatedSumFreeSize += levelNodeSize;
9723	++ctx.calculatedFreeCount;
9724	break;
9725	case Node::TYPE_ALLOCATION:
9726	++ctx.calculatedAllocationCount;
9727	if (!IsVirtual())
9728	{
9729	VMA_VALIDATE(curr->allocation.userData != VMA_NULL);
9730	}
9731	break;
9732	case Node::TYPE_SPLIT:
9733	{
9734	const uint32_t childrenLevel = level + `1`;
9735	const VkDeviceSize childrenLevelNodeSize = levelNodeSize >> `1`;
9736	const Node* const leftChild = curr->split.leftChild;
9737	VMA_VALIDATE(leftChild != VMA_NULL);
9738	VMA_VALIDATE(leftChild->offset == curr->offset);
9739	if (!ValidateNode(ctx, curr, leftChild, childrenLevel, childrenLevelNodeSize))
9740	{
9741	VMA_VALIDATE(false && "ValidateNode for left child failed.");
9742	}
9743	const Node* const rightChild = leftChild->buddy;
9744	VMA_VALIDATE(rightChild->offset == curr->offset + childrenLevelNodeSize);
9745	if (!ValidateNode(ctx, curr, rightChild, childrenLevel, childrenLevelNodeSize))
9746	{
9747	VMA_VALIDATE(false && "ValidateNode for right child failed.");
9748	}
9749	}
9750	break;
9751	default:
9752	return false;
9753	}
9754
9755	return true;
9756	}
9757
9758	uint32_t VmaBlockMetadata_Buddy::AllocSizeToLevel(VkDeviceSize allocSize) const
9759	{
9760	// I know this could be optimized somehow e.g. by using std::log2p1 from C++20.
9761	uint32_t level = `0`;
9762	VkDeviceSize currLevelNodeSize = m_UsableSize;
9763	VkDeviceSize nextLevelNodeSize = currLevelNodeSize >> `1`;
9764	while (allocSize <= nextLevelNodeSize && level + `1` < m_LevelCount)
9765	{
9766	++level;
9767	currLevelNodeSize >>= `1`;
9768	nextLevelNodeSize >>= `1`;
9769	}
9770	return level;
9771	}
9772
9773	void VmaBlockMetadata_Buddy::Free(VmaAllocHandle allocHandle)
9774	{
9775	uint32_t level = `0`;
9776	Node* node = FindAllocationNode((VkDeviceSize)allocHandle - `1`, level);
9777
9778	++m_FreeCount;
9779	--m_AllocationCount;
9780	m_SumFreeSize += LevelToNodeSize(level);
9781
9782	node->type = Node::TYPE_FREE;
9783
9784	// Join free nodes if possible.
9785	while (level > `0` && node->buddy->type == Node::TYPE_FREE)
9786	{
9787	RemoveFromFreeList(level, node->buddy);
9788	Node* const parent = node->parent;
9789
9790	m_NodeAllocator.Free(node->buddy);
9791	m_NodeAllocator.Free(node);
9792	parent->type = Node::TYPE_FREE;
9793
9794	node = parent;
9795	--level;
9796	--m_FreeCount;
9797	}
9798
9799	AddToFreeListFront(level, node);
9800	}
9801
9802	void VmaBlockMetadata_Buddy::AddNodeToDetailedStatistics(VmaDetailedStatistics& inoutStats, const Node* node, VkDeviceSize levelNodeSize) const
9803	{
9804	switch (node->type)
9805	{
9806	case Node::TYPE_FREE:
9807	VmaAddDetailedStatisticsUnusedRange(inoutStats, levelNodeSize);
9808	break;
9809	case Node::TYPE_ALLOCATION:
9810	VmaAddDetailedStatisticsAllocation(inoutStats, levelNodeSize);
9811	break;
9812	case Node::TYPE_SPLIT:
9813	{
9814	const VkDeviceSize childrenNodeSize = levelNodeSize / `2`;
9815	const Node* const leftChild = node->split.leftChild;
9816	AddNodeToDetailedStatistics(inoutStats, leftChild, childrenNodeSize);
9817	const Node* const rightChild = leftChild->buddy;
9818	AddNodeToDetailedStatistics(inoutStats, rightChild, childrenNodeSize);
9819	}
9820	break;
9821	default:
9822	VMA_ASSERT(`0`);
9823	}
9824	}
9825
9826	void VmaBlockMetadata_Buddy::AddToFreeListFront(uint32_t level, Node* node)
9827	{
9828	VMA_ASSERT(node->type == Node::TYPE_FREE);
9829
9830	// List is empty.
9831	Node* const frontNode = m_FreeList[level].front;
9832	if (frontNode == VMA_NULL)
9833	{
9834	VMA_ASSERT(m_FreeList[level].back == VMA_NULL);
9835	node->free.prev = node->free.next = VMA_NULL;
9836	m_FreeList[level].front = m_FreeList[level].back = node;
9837	}
9838	else
9839	{
9840	VMA_ASSERT(frontNode->free.prev == VMA_NULL);
9841	node->free.prev = VMA_NULL;
9842	node->free.next = frontNode;
9843	frontNode->free.prev = node;
9844	m_FreeList[level].front = node;
9845	}
9846	}
9847
9848	void VmaBlockMetadata_Buddy::RemoveFromFreeList(uint32_t level, Node* node)
9849	{
9850	VMA_ASSERT(m_FreeList[level].front != VMA_NULL);
9851
9852	// It is at the front.
9853	if (node->free.prev == VMA_NULL)
9854	{
9855	VMA_ASSERT(m_FreeList[level].front == node);
9856	m_FreeList[level].front = node->free.next;
9857	}
9858	else
9859	{
9860	Node* const prevFreeNode = node->free.prev;
9861	VMA_ASSERT(prevFreeNode->free.next == node);
9862	prevFreeNode->free.next = node->free.next;
9863	}
9864
9865	// It is at the back.
9866	if (node->free.next == VMA_NULL)
9867	{
9868	VMA_ASSERT(m_FreeList[level].back == node);
9869	m_FreeList[level].back = node->free.prev;
9870	}
9871	else
9872	{
9873	Node* const nextFreeNode = node->free.next;
9874	VMA_ASSERT(nextFreeNode->free.prev == node);
9875	nextFreeNode->free.prev = node->free.prev;
9876	}
9877	}
9878
9879	void VmaBlockMetadata_Buddy::DebugLogAllAllocationNode(Node* node, uint32_t level) const
9880	{
9881	switch (node->type)
9882	{
9883	case Node::TYPE_FREE:
9884	break;
9885	case Node::TYPE_ALLOCATION:
9886	DebugLogAllocation(node->offset, LevelToNodeSize(level), node->allocation.userData);
9887	break;
9888	case Node::TYPE_SPLIT:
9889	{
9890	++level;
9891	DebugLogAllAllocationNode(node->split.leftChild, level);
9892	DebugLogAllAllocationNode(node->split.leftChild->buddy, level);
9893	}
9894	break;
9895	default:
9896	VMA_ASSERT(`0`);
9897	}
9898	}
9899
9900	#if VMA_STATS_STRING_ENABLED
9901	void VmaBlockMetadata_Buddy::PrintDetailedMapNode(class VmaJsonWriter& json, const Node* node, VkDeviceSize levelNodeSize) const
9902	{
9903	switch (node->type)
9904	{
9905	case Node::TYPE_FREE:
9906	PrintDetailedMap_UnusedRange(json, node->offset, levelNodeSize);
9907	break;
9908	case Node::TYPE_ALLOCATION:
9909	PrintDetailedMap_Allocation(json, node->offset, levelNodeSize, node->allocation.userData);
9910	break;
9911	case Node::TYPE_SPLIT:
9912	{
9913	const VkDeviceSize childrenNodeSize = levelNodeSize / `2`;
9914	const Node* const leftChild = node->split.leftChild;
9915	PrintDetailedMapNode(json, leftChild, childrenNodeSize);
9916	const Node* const rightChild = leftChild->buddy;
9917	PrintDetailedMapNode(json, rightChild, childrenNodeSize);
9918	}
9919	break;
9920	default:
9921	VMA_ASSERT(`0`);
9922	}
9923	}
9924	#endif // VMA_STATS_STRING_ENABLED
9925	#endif // _VMA_BLOCK_METADATA_BUDDY_FUNCTIONS
9926	#endif // _VMA_BLOCK_METADATA_BUDDY
9927	#endif // #if 0
9928
9929	#ifndef _VMA_BLOCK_METADATA_TLSF
9930	// To not search current larger region if first allocation won't succeed and skip to smaller range
9931	// use with VMA_ALLOCATION_CREATE_STRATEGY_MIN_MEMORY_BIT as strategy in CreateAllocationRequest().
9932	// When fragmentation and reusal of previous blocks doesn't matter then use with
9933	// VMA_ALLOCATION_CREATE_STRATEGY_MIN_TIME_BIT for fastest alloc time possible.
9934	class VmaBlockMetadata_TLSF : public VmaBlockMetadata
9935	{
9936	VMA_CLASS_NO_COPY(VmaBlockMetadata_TLSF)
9937	public:
9938	VmaBlockMetadata_TLSF(const VkAllocationCallbacks* pAllocationCallbacks,
9939	VkDeviceSize bufferImageGranularity, bool isVirtual);
9940	virtual ~VmaBlockMetadata_TLSF();
9941
9942	size_t GetAllocationCount() const override { return m_AllocCount; }
9943	size_t GetFreeRegionsCount() const override { return m_BlocksFreeCount + `1`; }
9944	VkDeviceSize GetSumFreeSize() const override { return m_BlocksFreeSize + m_NullBlock->size; }
9945	bool IsEmpty() const override { return m_NullBlock->offset == `0`; }
9946	VkDeviceSize GetAllocationOffset(VmaAllocHandle allocHandle) const override { return ((Block*)allocHandle)->offset; };
9947
9948	void Init(VkDeviceSize size) override;
9949	bool Validate() const override;
9950
9951	void AddDetailedStatistics(VmaDetailedStatistics& inoutStats) const override;
9952	void AddStatistics(VmaStatistics& inoutStats) const override;
9953
9954	#if VMA_STATS_STRING_ENABLED
9955	void PrintDetailedMap(class VmaJsonWriter& json) const override;
9956	#endif
9957
9958	bool CreateAllocationRequest(
9959	VkDeviceSize allocSize,
9960	VkDeviceSize allocAlignment,
9961	bool upperAddress,
9962	VmaSuballocationType allocType,
9963	uint32_t strategy,
9964	VmaAllocationRequest* pAllocationRequest) override;
9965
9966	VkResult CheckCorruption(const void* pBlockData) override;
9967	void Alloc(
9968	const VmaAllocationRequest& request,
9969	VmaSuballocationType type,
9970	void* userData) override;
9971
9972	void Free(VmaAllocHandle allocHandle) override;
9973	void GetAllocationInfo(VmaAllocHandle allocHandle, VmaVirtualAllocationInfo& outInfo) override;
9974	void* GetAllocationUserData(VmaAllocHandle allocHandle) const override;
9975	VmaAllocHandle GetAllocationListBegin() const override;
9976	VmaAllocHandle GetNextAllocation(VmaAllocHandle prevAlloc) const override;
9977	VkDeviceSize GetNextFreeRegionSize(VmaAllocHandle alloc) const override;
9978	void Clear() override;
9979	void SetAllocationUserData(VmaAllocHandle allocHandle, void* userData) override;
9980	void DebugLogAllAllocations() const override;
9981
9982	private:
9983	// According to original paper it should be preferable 4 or 5:
9984	// M. Masmano, I. Ripoll, A. Crespo, and J. Real "TLSF: a New Dynamic Memory Allocator for Real-Time Systems"
9985	// http://www.gii.upv.es/tlsf/files/ecrts04_tlsf.pdf
9986	static const uint8_t SECOND_LEVEL_INDEX = `5`;
9987	static const uint16_t SMALL_BUFFER_SIZE = `256`;
9988	static const uint32_t INITIAL_BLOCK_ALLOC_COUNT = `16`;
9989	static const uint8_t MEMORY_CLASS_SHIFT = `7`;
9990	static const uint8_t MAX_MEMORY_CLASSES = `65` - MEMORY_CLASS_SHIFT;
9991
9992	class Block
9993	{
9994	public:
9995	VkDeviceSize offset;
9996	VkDeviceSize size;
9997	Block* prevPhysical;
9998	Block* nextPhysical;
9999
10000	void MarkFree() { prevFree = VMA_NULL; }
10001	void MarkTaken() { prevFree = this; }
10002	bool IsFree() const { return prevFree != this; }
10003	void& UserData() { VMA_HEAVY_ASSERT(!IsFree()); return* userData; }
10004	Block& PrevFree() { return* prevFree; }
10005	Block& NextFree() { VMA_HEAVY_ASSERT(IsFree()); return* nextFree; }
10006
10007	private:
10008	Block* prevFree; // Address of the same block here indicates that block is taken
10009	union
10010	{
10011	Block* nextFree;
10012	void* userData;
10013	};
10014	};
10015
10016	size_t m_AllocCount;
10017	// Total number of free blocks besides null block
10018	size_t m_BlocksFreeCount;
10019	// Total size of free blocks excluding null block
10020	VkDeviceSize m_BlocksFreeSize;
10021	uint32_t m_IsFreeBitmap;
10022	uint8_t m_MemoryClasses;
10023	uint32_t m_InnerIsFreeBitmap[MAX_MEMORY_CLASSES];
10024	uint32_t m_ListsCount;
10025	/*
10026	* 0: 0-3 lists for small buffers
10027	* 1+: 0-(2^SLI-1) lists for normal buffers
10028	*/
10029	Block** m_FreeList;
10030	VmaPoolAllocator<Block> m_BlockAllocator;
10031	Block* m_NullBlock;
10032	VmaBlockBufferImageGranularity m_GranularityHandler;
10033
10034	uint8_t SizeToMemoryClass(VkDeviceSize size) const;
10035	uint16_t SizeToSecondIndex(VkDeviceSize size, uint8_t memoryClass) const;
10036	uint32_t GetListIndex(uint8_t memoryClass, uint16_t secondIndex) const;
10037	uint32_t GetListIndex(VkDeviceSize size) const;
10038
10039	void RemoveFreeBlock(Block* block);
10040	void InsertFreeBlock(Block* block);
10041	void MergeBlock(Block* block, Block* prev);
10042
10043	Block* FindFreeBlock(VkDeviceSize size, uint32_t& listIndex) const;
10044	bool CheckBlock(
10045	Block& block,
10046	uint32_t listIndex,
10047	VkDeviceSize allocSize,
10048	VkDeviceSize allocAlignment,
10049	VmaSuballocationType allocType,
10050	VmaAllocationRequest* pAllocationRequest);
10051	};
10052
10053	#ifndef _VMA_BLOCK_METADATA_TLSF_FUNCTIONS
10054	VmaBlockMetadata_TLSF::VmaBlockMetadata_TLSF(const VkAllocationCallbacks* pAllocationCallbacks,
10055	VkDeviceSize bufferImageGranularity, bool isVirtual)
10056	: VmaBlockMetadata(pAllocationCallbacks, bufferImageGranularity, isVirtual),
10057	m_AllocCount(`0`),
10058	m_BlocksFreeCount(`0`),
10059	m_BlocksFreeSize(`0`),
10060	m_IsFreeBitmap(`0`),
10061	m_MemoryClasses(`0`),
10062	m_ListsCount(`0`),
10063	m_FreeList(VMA_NULL),
10064	m_BlockAllocator(pAllocationCallbacks, INITIAL_BLOCK_ALLOC_COUNT),
10065	m_NullBlock(VMA_NULL),
10066	m_GranularityHandler(bufferImageGranularity) {}
10067
10068	VmaBlockMetadata_TLSF::~VmaBlockMetadata_TLSF()
10069	{
10070	if (m_FreeList)
10071	vma_delete_array(GetAllocationCallbacks(), m_FreeList, m_ListsCount);
10072	m_GranularityHandler.Destroy(GetAllocationCallbacks());
10073	}
10074
10075	void VmaBlockMetadata_TLSF::Init(VkDeviceSize size)
10076	{
10077	VmaBlockMetadata::Init(size);
10078
10079	if (!IsVirtual())
10080	m_GranularityHandler.Init(GetAllocationCallbacks(), size);
10081
10082	m_NullBlock = m_BlockAllocator.Alloc();
10083	m_NullBlock->size = size;
10084	m_NullBlock->offset = `0`;
10085	m_NullBlock->prevPhysical = VMA_NULL;
10086	m_NullBlock->nextPhysical = VMA_NULL;
10087	m_NullBlock->MarkFree();
10088	m_NullBlock->NextFree() = VMA_NULL;
10089	m_NullBlock->PrevFree() = VMA_NULL;
10090	uint8_t memoryClass = SizeToMemoryClass(size);
10091	uint16_t sli = SizeToSecondIndex(size, memoryClass);
10092	m_ListsCount = (memoryClass == `0` ? `0` : (memoryClass - `1`) * (`1UL` << SECOND_LEVEL_INDEX) + sli) + `1`;
10093	if (IsVirtual())
10094	m_ListsCount += `1UL` << SECOND_LEVEL_INDEX;
10095	else
10096	m_ListsCount += `4`;
10097
10098	m_MemoryClasses = memoryClass + `2`;
10099	memset(m_InnerIsFreeBitmap, `0`, MAX_MEMORY_CLASSES * sizeof(uint32_t));
10100
10101	m_FreeList = vma_new_array(GetAllocationCallbacks(), Block*, m_ListsCount);
10102	memset(m_FreeList, `0`, m_ListsCount * sizeof(Block*));
10103	}
10104
10105	bool VmaBlockMetadata_TLSF::Validate() const
10106	{
10107	VMA_VALIDATE(GetSumFreeSize() <= GetSize());
10108
10109	VkDeviceSize calculatedSize = m_NullBlock->size;
10110	VkDeviceSize calculatedFreeSize = m_NullBlock->size;
10111	size_t allocCount = `0`;
10112	size_t freeCount = `0`;
10113
10114	// Check integrity of free lists
10115	for (uint32_t list = `0`; list < m_ListsCount; ++list)
10116	{
10117	Block* block = m_FreeList[list];
10118	if (block != VMA_NULL)
10119	{
10120	VMA_VALIDATE(block->IsFree());
10121	VMA_VALIDATE(block->PrevFree() == VMA_NULL);
10122	while (block->NextFree())
10123	{
10124	VMA_VALIDATE(block->NextFree()->IsFree());
10125	VMA_VALIDATE(block->NextFree()->PrevFree() == block);
10126	block = block->NextFree();
10127	}
10128	}
10129	}
10130
10131	VkDeviceSize nextOffset = m_NullBlock->offset;
10132	auto validateCtx = m_GranularityHandler.StartValidation(GetAllocationCallbacks(), IsVirtual());
10133
10134	VMA_VALIDATE(m_NullBlock->nextPhysical == VMA_NULL);
10135	if (m_NullBlock->prevPhysical)
10136	{
10137	VMA_VALIDATE(m_NullBlock->prevPhysical->nextPhysical == m_NullBlock);
10138	}
10139	// Check all blocks
10140	for (Block* prev = m_NullBlock->prevPhysical; prev != VMA_NULL; prev = prev->prevPhysical)
10141	{
10142	VMA_VALIDATE(prev->offset + prev->size == nextOffset);
10143	nextOffset = prev->offset;
10144	calculatedSize += prev->size;
10145
10146	uint32_t listIndex = GetListIndex(prev->size);
10147	if (prev->IsFree())
10148	{
10149	++freeCount;
10150	// Check if free block belongs to free list
10151	Block* freeBlock = m_FreeList[listIndex];
10152	VMA_VALIDATE(freeBlock != VMA_NULL);
10153
10154	bool found = false;
10155	do
10156	{
10157	if (freeBlock == prev)
10158	found = true;
10159
10160	freeBlock = freeBlock->NextFree();
10161	} while (!found && freeBlock != VMA_NULL);
10162
10163	VMA_VALIDATE(found);
10164	calculatedFreeSize += prev->size;
10165	}
10166	else
10167	{
10168	++allocCount;
10169	// Check if taken block is not on a free list
10170	Block* freeBlock = m_FreeList[listIndex];
10171	while (freeBlock)
10172	{
10173	VMA_VALIDATE(freeBlock != prev);
10174	freeBlock = freeBlock->NextFree();
10175	}
10176
10177	if (!IsVirtual())
10178	{
10179	VMA_VALIDATE(m_GranularityHandler.Validate(validateCtx, prev->offset, prev->size));
10180	}
10181	}
10182
10183	if (prev->prevPhysical)
10184	{
10185	VMA_VALIDATE(prev->prevPhysical->nextPhysical == prev);
10186	}
10187	}
10188
10189	if (!IsVirtual())
10190	{
10191	VMA_VALIDATE(m_GranularityHandler.FinishValidation(validateCtx));
10192	}
10193
10194	VMA_VALIDATE(nextOffset == `0`);
10195	VMA_VALIDATE(calculatedSize == GetSize());
10196	VMA_VALIDATE(calculatedFreeSize == GetSumFreeSize());
10197	VMA_VALIDATE(allocCount == m_AllocCount);
10198	VMA_VALIDATE(freeCount == m_BlocksFreeCount);
10199
10200	return true;
10201	}
10202
10203	void VmaBlockMetadata_TLSF::AddDetailedStatistics(VmaDetailedStatistics& inoutStats) const
10204	{
10205	inoutStats.statistics.blockCount++;
10206	inoutStats.statistics.blockBytes += GetSize();
10207	if (m_NullBlock->size > `0`)
10208	VmaAddDetailedStatisticsUnusedRange(inoutStats, m_NullBlock->size);
10209
10210	for (Block* block = m_NullBlock->prevPhysical; block != VMA_NULL; block = block->prevPhysical)
10211	{
10212	if (block->IsFree())
10213	VmaAddDetailedStatisticsUnusedRange(inoutStats, block->size);
10214	else
10215	VmaAddDetailedStatisticsAllocation(inoutStats, block->size);
10216	}
10217	}
10218
10219	void VmaBlockMetadata_TLSF::AddStatistics(VmaStatistics& inoutStats) const
10220	{
10221	inoutStats.blockCount++;
10222	inoutStats.allocationCount += (uint32_t)m_AllocCount;
10223	inoutStats.blockBytes += GetSize();
10224	inoutStats.allocationBytes += GetSize() - GetSumFreeSize();
10225	}
10226
10227	#if VMA_STATS_STRING_ENABLED
10228	void VmaBlockMetadata_TLSF::PrintDetailedMap(class VmaJsonWriter& json) const
10229	{
10230	size_t blockCount = m_AllocCount + m_BlocksFreeCount;
10231	VmaStlAllocator<Block*> allocator(GetAllocationCallbacks());
10232	VmaVector<Block, VmaStlAllocator<Block>> blockList(blockCount, allocator);
10233
10234	size_t i = blockCount;
10235	for (Block* block = m_NullBlock->prevPhysical; block != VMA_NULL; block = block->prevPhysical)
10236	{
10237	blockList[--i] = block;
10238	}
10239	VMA_ASSERT(i == `0`);
10240
10241	VmaDetailedStatistics stats;
10242	VmaClearDetailedStatistics(stats);
10243	AddDetailedStatistics(stats);
10244
10245	PrintDetailedMap_Begin(json,
10246	stats.statistics.blockBytes - stats.statistics.allocationBytes,
10247	stats.statistics.allocationCount,
10248	stats.unusedRangeCount);
10249
10250	for (; i < blockCount; ++i)
10251	{
10252	Block* block = blockList[i];
10253	if (block->IsFree())
10254	PrintDetailedMap_UnusedRange(json, block->offset, block->size);
10255	else
10256	PrintDetailedMap_Allocation(json, block->offset, block->size, block->UserData());
10257	}
10258	if (m_NullBlock->size > `0`)
10259	PrintDetailedMap_UnusedRange(json, m_NullBlock->offset, m_NullBlock->size);
10260
10261	PrintDetailedMap_End(json);
10262	}
10263	#endif
10264
10265	bool VmaBlockMetadata_TLSF::CreateAllocationRequest(
10266	VkDeviceSize allocSize,
10267	VkDeviceSize allocAlignment,
10268	bool upperAddress,
10269	VmaSuballocationType allocType,
10270	uint32_t strategy,
10271	VmaAllocationRequest* pAllocationRequest)
10272	{
10273	VMA_ASSERT(allocSize > `0` && "Cannot allocate empty block!");
10274	VMA_ASSERT(!upperAddress && "VMA_ALLOCATION_CREATE_UPPER_ADDRESS_BIT can be used only with linear algorithm.");
10275
10276	// For small granularity round up
10277	if (!IsVirtual())
10278	m_GranularityHandler.RoundupAllocRequest(allocType, allocSize, allocAlignment);
10279
10280	allocSize += GetDebugMargin();
10281	// Quick check for too small pool
10282	if (allocSize > GetSumFreeSize())
10283	return false;
10284
10285	// If no free blocks in pool then check only null block
10286	if (m_BlocksFreeCount == `0`)
10287	return CheckBlock(*m_NullBlock, m_ListsCount, allocSize, allocAlignment, allocType, pAllocationRequest);
10288
10289	// Round up to the next block
10290	VkDeviceSize sizeForNextList = allocSize;
10291	VkDeviceSize smallSizeStep = SMALL_BUFFER_SIZE / (IsVirtual() ? `1` << SECOND_LEVEL_INDEX : `4`);
10292	if (allocSize > SMALL_BUFFER_SIZE)
10293	{
10294	sizeForNextList += (`1ULL` << (VMA_BITSCAN_MSB(allocSize) - SECOND_LEVEL_INDEX));
10295	}
10296	else if (allocSize > SMALL_BUFFER_SIZE - smallSizeStep)
10297	sizeForNextList = SMALL_BUFFER_SIZE + `1`;
10298	else
10299	sizeForNextList += smallSizeStep;
10300
10301	uint32_t nextListIndex = `0`;
10302	uint32_t prevListIndex = `0`;
10303	Block* nextListBlock = VMA_NULL;
10304	Block* prevListBlock = VMA_NULL;
10305
10306	// Check blocks according to strategies
10307	if (strategy & VMA_ALLOCATION_CREATE_STRATEGY_MIN_TIME_BIT)
10308	{
10309	// Quick check for larger block first
10310	nextListBlock = FindFreeBlock(sizeForNextList, nextListIndex);
10311	if (nextListBlock != VMA_NULL && CheckBlock(*nextListBlock, nextListIndex, allocSize, allocAlignment, allocType, pAllocationRequest))
10312	return true;
10313
10314	// If not fitted then null block
10315	if (CheckBlock(*m_NullBlock, m_ListsCount, allocSize, allocAlignment, allocType, pAllocationRequest))
10316	return true;
10317
10318	// Null block failed, search larger bucket
10319	while (nextListBlock)
10320	{
10321	if (CheckBlock(*nextListBlock, nextListIndex, allocSize, allocAlignment, allocType, pAllocationRequest))
10322	return true;
10323	nextListBlock = nextListBlock->NextFree();
10324	}
10325
10326	// Failed again, check best fit bucket
10327	prevListBlock = FindFreeBlock(allocSize, prevListIndex);
10328	while (prevListBlock)
10329	{
10330	if (CheckBlock(*prevListBlock, prevListIndex, allocSize, allocAlignment, allocType, pAllocationRequest))
10331	return true;
10332	prevListBlock = prevListBlock->NextFree();
10333	}
10334	}
10335	else if (strategy & VMA_ALLOCATION_CREATE_STRATEGY_MIN_MEMORY_BIT)
10336	{
10337	// Check best fit bucket
10338	prevListBlock = FindFreeBlock(allocSize, prevListIndex);
10339	while (prevListBlock)
10340	{
10341	if (CheckBlock(*prevListBlock, prevListIndex, allocSize, allocAlignment, allocType, pAllocationRequest))
10342	return true;
10343	prevListBlock = prevListBlock->NextFree();
10344	}
10345
10346	// If failed check null block
10347	if (CheckBlock(*m_NullBlock, m_ListsCount, allocSize, allocAlignment, allocType, pAllocationRequest))
10348	return true;
10349
10350	// Check larger bucket
10351	nextListBlock = FindFreeBlock(sizeForNextList, nextListIndex);
10352	while (nextListBlock)
10353	{
10354	if (CheckBlock(*nextListBlock, nextListIndex, allocSize, allocAlignment, allocType, pAllocationRequest))
10355	return true;
10356	nextListBlock = nextListBlock->NextFree();
10357	}
10358	}
10359	else if (strategy & VMA_ALLOCATION_CREATE_STRATEGY_MIN_OFFSET_BIT )
10360	{
10361	// Perform search from the start
10362	VmaStlAllocator<Block*> allocator(GetAllocationCallbacks());
10363	VmaVector<Block, VmaStlAllocator<Block>> blockList(m_BlocksFreeCount, allocator);
10364
10365	size_t i = m_BlocksFreeCount;
10366	for (Block* block = m_NullBlock->prevPhysical; block != VMA_NULL; block = block->prevPhysical)
10367	{
10368	if (block->IsFree() && block->size >= allocSize)
10369	blockList[--i] = block;
10370	}
10371
10372	for (; i < m_BlocksFreeCount; ++i)
10373	{
10374	Block& block = *blockList[i];
10375	if (CheckBlock(block, GetListIndex(block.size), allocSize, allocAlignment, allocType, pAllocationRequest))
10376	return true;
10377	}
10378
10379	// If failed check null block
10380	if (CheckBlock(*m_NullBlock, m_ListsCount, allocSize, allocAlignment, allocType, pAllocationRequest))
10381	return true;
10382
10383	// Whole range searched, no more memory
10384	return false;
10385	}
10386	else
10387	{
10388	// Check larger bucket
10389	nextListBlock = FindFreeBlock(sizeForNextList, nextListIndex);
10390	while (nextListBlock)
10391	{
10392	if (CheckBlock(*nextListBlock, nextListIndex, allocSize, allocAlignment, allocType, pAllocationRequest))
10393	return true;
10394	nextListBlock = nextListBlock->NextFree();
10395	}
10396
10397	// If failed check null block
10398	if (CheckBlock(*m_NullBlock, m_ListsCount, allocSize, allocAlignment, allocType, pAllocationRequest))
10399	return true;
10400
10401	// Check best fit bucket
10402	prevListBlock = FindFreeBlock(allocSize, prevListIndex);
10403	while (prevListBlock)
10404	{
10405	if (CheckBlock(*prevListBlock, prevListIndex, allocSize, allocAlignment, allocType, pAllocationRequest))
10406	return true;
10407	prevListBlock = prevListBlock->NextFree();
10408	}
10409	}
10410
10411	// Worst case, full search has to be done
10412	while (++nextListIndex < m_ListsCount)
10413	{
10414	nextListBlock = m_FreeList[nextListIndex];
10415	while (nextListBlock)
10416	{
10417	if (CheckBlock(*nextListBlock, nextListIndex, allocSize, allocAlignment, allocType, pAllocationRequest))
10418	return true;
10419	nextListBlock = nextListBlock->NextFree();
10420	}
10421	}
10422
10423	// No more memory sadly
10424	return false;
10425	}
10426
10427	VkResult VmaBlockMetadata_TLSF::CheckCorruption(const void* pBlockData)
10428	{
10429	for (Block* block = m_NullBlock->prevPhysical; block != VMA_NULL; block = block->prevPhysical)
10430	{
10431	if (!block->IsFree())
10432	{
10433	if (!VmaValidateMagicValue(pBlockData, block->offset + block->size))
10434	{
10435	VMA_ASSERT(`0` && "MEMORY CORRUPTION DETECTED AFTER VALIDATED ALLOCATION!");
10436	return VK_ERROR_UNKNOWN_COPY;
10437	}
10438	}
10439	}
10440
10441	return VK_SUCCESS;
10442	}
10443
10444	void VmaBlockMetadata_TLSF::Alloc(
10445	const VmaAllocationRequest& request,
10446	VmaSuballocationType type,
10447	void* userData)
10448	{
10449	VMA_ASSERT(request.type == VmaAllocationRequestType::TLSF);
10450
10451	// Get block and pop it from the free list
10452	Block* currentBlock = (Block*)request.allocHandle;
10453	VkDeviceSize offset = request.algorithmData;
10454	VMA_ASSERT(currentBlock != VMA_NULL);
10455	VMA_ASSERT(currentBlock->offset <= offset);
10456
10457	if (currentBlock != m_NullBlock)
10458	RemoveFreeBlock(currentBlock);
10459
10460	VkDeviceSize debugMargin = GetDebugMargin();
10461	VkDeviceSize misssingAlignment = offset - currentBlock->offset;
10462
10463	// Append missing alignment to prev block or create new one
10464	if (misssingAlignment)
10465	{
10466	Block* prevBlock = currentBlock->prevPhysical;
10467	VMA_ASSERT(prevBlock != VMA_NULL && "There should be no missing alignment at offset 0!");
10468
10469	if (prevBlock->IsFree() && prevBlock->size != debugMargin)
10470	{
10471	uint32_t oldList = GetListIndex(prevBlock->size);
10472	prevBlock->size += misssingAlignment;
10473	// Check if new size crosses list bucket
10474	if (oldList != GetListIndex(prevBlock->size))
10475	{
10476	prevBlock->size -= misssingAlignment;
10477	RemoveFreeBlock(prevBlock);
10478	prevBlock->size += misssingAlignment;
10479	InsertFreeBlock(prevBlock);
10480	}
10481	else
10482	m_BlocksFreeSize += misssingAlignment;
10483	}
10484	else
10485	{
10486	Block* newBlock = m_BlockAllocator.Alloc();
10487	currentBlock->prevPhysical = newBlock;
10488	prevBlock->nextPhysical = newBlock;
10489	newBlock->prevPhysical = prevBlock;
10490	newBlock->nextPhysical = currentBlock;
10491	newBlock->size = misssingAlignment;
10492	newBlock->offset = currentBlock->offset;
10493	newBlock->MarkTaken();
10494
10495	InsertFreeBlock(newBlock);
10496	}
10497
10498	currentBlock->size -= misssingAlignment;
10499	currentBlock->offset += misssingAlignment;
10500	}
10501
10502	VkDeviceSize size = request.size + debugMargin;
10503	if (currentBlock->size == size)
10504	{
10505	if (currentBlock == m_NullBlock)
10506	{
10507	// Setup new null block
10508	m_NullBlock = m_BlockAllocator.Alloc();
10509	m_NullBlock->size = `0`;
10510	m_NullBlock->offset = currentBlock->offset + size;
10511	m_NullBlock->prevPhysical = currentBlock;
10512	m_NullBlock->nextPhysical = VMA_NULL;
10513	m_NullBlock->MarkFree();
10514	m_NullBlock->PrevFree() = VMA_NULL;
10515	m_NullBlock->NextFree() = VMA_NULL;
10516	currentBlock->nextPhysical = m_NullBlock;
10517	currentBlock->MarkTaken();
10518	}
10519	}
10520	else
10521	{
10522	VMA_ASSERT(currentBlock->size > size && "Proper block already found, shouldn't find smaller one!");
10523
10524	// Create new free block
10525	Block* newBlock = m_BlockAllocator.Alloc();
10526	newBlock->size = currentBlock->size - size;
10527	newBlock->offset = currentBlock->offset + size;
10528	newBlock->prevPhysical = currentBlock;
10529	newBlock->nextPhysical = currentBlock->nextPhysical;
10530	currentBlock->nextPhysical = newBlock;
10531	currentBlock->size = size;
10532
10533	if (currentBlock == m_NullBlock)
10534	{
10535	m_NullBlock = newBlock;
10536	m_NullBlock->MarkFree();
10537	m_NullBlock->NextFree() = VMA_NULL;
10538	m_NullBlock->PrevFree() = VMA_NULL;
10539	currentBlock->MarkTaken();
10540	}
10541	else
10542	{
10543	newBlock->nextPhysical->prevPhysical = newBlock;
10544	newBlock->MarkTaken();
10545	InsertFreeBlock(newBlock);
10546	}
10547	}
10548	currentBlock->UserData() = userData;
10549
10550	if (debugMargin > `0`)
10551	{
10552	currentBlock->size -= debugMargin;
10553	Block* newBlock = m_BlockAllocator.Alloc();
10554	newBlock->size = debugMargin;
10555	newBlock->offset = currentBlock->offset + currentBlock->size;
10556	newBlock->prevPhysical = currentBlock;
10557	newBlock->nextPhysical = currentBlock->nextPhysical;
10558	newBlock->MarkTaken();
10559	currentBlock->nextPhysical->prevPhysical = newBlock;
10560	currentBlock->nextPhysical = newBlock;
10561	InsertFreeBlock(newBlock);
10562	}
10563
10564	if (!IsVirtual())
10565	m_GranularityHandler.AllocPages((uint8_t)(uintptr_t)request.customData,
10566	currentBlock->offset, currentBlock->size);
10567	++m_AllocCount;
10568	}
10569
10570	void VmaBlockMetadata_TLSF::Free(VmaAllocHandle allocHandle)
10571	{
10572	Block* block = (Block*)allocHandle;
10573	Block* next = block->nextPhysical;
10574	VMA_ASSERT(!block->IsFree() && "Block is already free!");
10575
10576	if (!IsVirtual())
10577	m_GranularityHandler.FreePages(block->offset, block->size);
10578	--m_AllocCount;
10579
10580	VkDeviceSize debugMargin = GetDebugMargin();
10581	if (debugMargin > `0`)
10582	{
10583	RemoveFreeBlock(next);
10584	MergeBlock(next, block);
10585	block = next;
10586	next = next->nextPhysical;
10587	}
10588
10589	// Try merging
10590	Block* prev = block->prevPhysical;
10591	if (prev != VMA_NULL && prev->IsFree() && prev->size != debugMargin)
10592	{
10593	RemoveFreeBlock(prev);
10594	MergeBlock(block, prev);
10595	}
10596
10597	if (!next->IsFree())
10598	InsertFreeBlock(block);
10599	else if (next == m_NullBlock)
10600	MergeBlock(m_NullBlock, block);
10601	else
10602	{
10603	RemoveFreeBlock(next);
10604	MergeBlock(next, block);
10605	InsertFreeBlock(next);
10606	}
10607	}
10608
10609	void VmaBlockMetadata_TLSF::GetAllocationInfo(VmaAllocHandle allocHandle, VmaVirtualAllocationInfo& outInfo)
10610	{
10611	Block* block = (Block*)allocHandle;
10612	VMA_ASSERT(!block->IsFree() && "Cannot get allocation info for free block!");
10613	outInfo.offset = block->offset;
10614	outInfo.size = block->size;
10615	outInfo.pUserData = block->UserData();
10616	}
10617
10618	void* VmaBlockMetadata_TLSF::GetAllocationUserData(VmaAllocHandle allocHandle) const
10619	{
10620	Block* block = (Block*)allocHandle;
10621	VMA_ASSERT(!block->IsFree() && "Cannot get user data for free block!");
10622	return block->UserData();
10623	}
10624
10625	VmaAllocHandle VmaBlockMetadata_TLSF::GetAllocationListBegin() const
10626	{
10627	if (m_AllocCount == `0`)
10628	return VK_NULL_HANDLE;
10629
10630	for (Block* block = m_NullBlock->prevPhysical; block; block = block->prevPhysical)
10631	{
10632	if (!block->IsFree())
10633	return (VmaAllocHandle)block;
10634	}
10635	VMA_ASSERT(false && "If m_AllocCount > 0 then should find any allocation!");
10636	return VK_NULL_HANDLE;
10637	}
10638
10639	VmaAllocHandle VmaBlockMetadata_TLSF::GetNextAllocation(VmaAllocHandle prevAlloc) const
10640	{
10641	Block* startBlock = (Block*)prevAlloc;
10642	VMA_ASSERT(!startBlock->IsFree() && "Incorrect block!");
10643
10644	for (Block* block = startBlock->prevPhysical; block; block = block->prevPhysical)
10645	{
10646	if (!block->IsFree())
10647	return (VmaAllocHandle)block;
10648	}
10649	return VK_NULL_HANDLE;
10650	}
10651
10652	VkDeviceSize VmaBlockMetadata_TLSF::GetNextFreeRegionSize(VmaAllocHandle alloc) const
10653	{
10654	Block* block = (Block*)alloc;
10655	VMA_ASSERT(!block->IsFree() && "Incorrect block!");
10656
10657	if (block->prevPhysical)
10658	return block->prevPhysical->IsFree() ? block->prevPhysical->size : `0`;
10659	return `0`;
10660	}
10661
10662	void VmaBlockMetadata_TLSF::Clear()
10663	{
10664	m_AllocCount = `0`;
10665	m_BlocksFreeCount = `0`;
10666	m_BlocksFreeSize = `0`;
10667	m_IsFreeBitmap = `0`;
10668	m_NullBlock->offset = `0`;
10669	m_NullBlock->size = GetSize();
10670	Block* block = m_NullBlock->prevPhysical;
10671	m_NullBlock->prevPhysical = VMA_NULL;
10672	while (block)
10673	{
10674	Block* prev = block->prevPhysical;
10675	m_BlockAllocator.Free(block);
10676	block = prev;
10677	}
10678	memset(m_FreeList, `0`, m_ListsCount * sizeof(Block*));
10679	memset(m_InnerIsFreeBitmap, `0`, m_MemoryClasses * sizeof(uint32_t));
10680	m_GranularityHandler.Clear();
10681	}
10682
10683	void VmaBlockMetadata_TLSF::SetAllocationUserData(VmaAllocHandle allocHandle, void* userData)
10684	{
10685	Block* block = (Block*)allocHandle;
10686	VMA_ASSERT(!block->IsFree() && "Trying to set user data for not allocated block!");
10687	block->UserData() = userData;
10688	}
10689
10690	void VmaBlockMetadata_TLSF::DebugLogAllAllocations() const
10691	{
10692	for (Block* block = m_NullBlock->prevPhysical; block != VMA_NULL; block = block->prevPhysical)
10693	if (!block->IsFree())
10694	DebugLogAllocation(block->offset, block->size, block->UserData());
10695	}
10696
10697	uint8_t VmaBlockMetadata_TLSF::SizeToMemoryClass(VkDeviceSize size) const
10698	{
10699	if (size > SMALL_BUFFER_SIZE)
10700	return VMA_BITSCAN_MSB(size) - MEMORY_CLASS_SHIFT;
10701	return `0`;
10702	}
10703
10704	uint16_t VmaBlockMetadata_TLSF::SizeToSecondIndex(VkDeviceSize size, uint8_t memoryClass) const
10705	{
10706	if (memoryClass == `0`)
10707	{
10708	if (IsVirtual())
10709	return static_cast<uint16_t>((size - `1`) / `8`);
10710	else
10711	return static_cast<uint16_t>((size - `1`) / `64`);
10712	}
10713	return static_cast<uint16_t>((size >> (memoryClass + MEMORY_CLASS_SHIFT - SECOND_LEVEL_INDEX)) ^ (`1U` << SECOND_LEVEL_INDEX));
10714	}
10715
10716	uint32_t VmaBlockMetadata_TLSF::GetListIndex(uint8_t memoryClass, uint16_t secondIndex) const
10717	{
10718	if (memoryClass == `0`)
10719	return secondIndex;
10720
10721	const uint32_t index = static_cast<uint32_t>(memoryClass - `1`) * (`1` << SECOND_LEVEL_INDEX) + secondIndex;
10722	if (IsVirtual())
10723	return index + (`1` << SECOND_LEVEL_INDEX);
10724	else
10725	return index + `4`;
10726	}
10727
10728	uint32_t VmaBlockMetadata_TLSF::GetListIndex(VkDeviceSize size) const
10729	{
10730	uint8_t memoryClass = SizeToMemoryClass(size);
10731	return GetListIndex(memoryClass, SizeToSecondIndex(size, memoryClass));
10732	}
10733
10734	void VmaBlockMetadata_TLSF::RemoveFreeBlock(Block* block)
10735	{
10736	VMA_ASSERT(block != m_NullBlock);
10737	VMA_ASSERT(block->IsFree());
10738
10739	if (block->NextFree() != VMA_NULL)
10740	block->NextFree()->PrevFree() = block->PrevFree();
10741	if (block->PrevFree() != VMA_NULL)
10742	block->PrevFree()->NextFree() = block->NextFree();
10743	else
10744	{
10745	uint8_t memClass = SizeToMemoryClass(block->size);
10746	uint16_t secondIndex = SizeToSecondIndex(block->size, memClass);
10747	uint32_t index = GetListIndex(memClass, secondIndex);
10748	VMA_ASSERT(m_FreeList[index] == block);
10749	m_FreeList[index] = block->NextFree();
10750	if (block->NextFree() == VMA_NULL)
10751	{
10752	m_InnerIsFreeBitmap[memClass] &= ~(`1U` << secondIndex);
10753	if (m_InnerIsFreeBitmap[memClass] == `0`)
10754	m_IsFreeBitmap &= ~(`1UL` << memClass);
10755	}
10756	}
10757	block->MarkTaken();
10758	block->UserData() = VMA_NULL;
10759	--m_BlocksFreeCount;
10760	m_BlocksFreeSize -= block->size;
10761	}
10762
10763	void VmaBlockMetadata_TLSF::InsertFreeBlock(Block* block)
10764	{
10765	VMA_ASSERT(block != m_NullBlock);
10766	VMA_ASSERT(!block->IsFree() && "Cannot insert block twice!");
10767
10768	uint8_t memClass = SizeToMemoryClass(block->size);
10769	uint16_t secondIndex = SizeToSecondIndex(block->size, memClass);
10770	uint32_t index = GetListIndex(memClass, secondIndex);
10771	VMA_ASSERT(index < m_ListsCount);
10772	block->PrevFree() = VMA_NULL;
10773	block->NextFree() = m_FreeList[index];
10774	m_FreeList[index] = block;
10775	if (block->NextFree() != VMA_NULL)
10776	block->NextFree()->PrevFree() = block;
10777	else
10778	{
10779	m_InnerIsFreeBitmap[memClass] \|= `1U` << secondIndex;
10780	m_IsFreeBitmap \|= `1UL` << memClass;
10781	}
10782	++m_BlocksFreeCount;
10783	m_BlocksFreeSize += block->size;
10784	}
10785
10786	void VmaBlockMetadata_TLSF::MergeBlock(Block* block, Block* prev)
10787	{
10788	VMA_ASSERT(block->prevPhysical == prev && "Cannot merge seperate physical regions!");
10789	VMA_ASSERT(!prev->IsFree() && "Cannot merge block that belongs to free list!");
10790
10791	block->offset = prev->offset;
10792	block->size += prev->size;
10793	block->prevPhysical = prev->prevPhysical;
10794	if (block->prevPhysical)
10795	block->prevPhysical->nextPhysical = block;
10796	m_BlockAllocator.Free(prev);
10797	}
10798
10799	VmaBlockMetadata_TLSF::Block* VmaBlockMetadata_TLSF::FindFreeBlock(VkDeviceSize size, uint32_t& listIndex) const
10800	{
10801	uint8_t memoryClass = SizeToMemoryClass(size);
10802	uint32_t innerFreeMap = m_InnerIsFreeBitmap[memoryClass] & (~`0U` << SizeToSecondIndex(size, memoryClass));
10803	if (!innerFreeMap)
10804	{
10805	// Check higher levels for avaiable blocks
10806	uint32_t freeMap = m_IsFreeBitmap & (~`0UL` << (memoryClass + `1`));
10807	if (!freeMap)
10808	return VMA_NULL; // No more memory avaible
10809
10810	// Find lowest free region
10811	memoryClass = VMA_BITSCAN_LSB(freeMap);
10812	innerFreeMap = m_InnerIsFreeBitmap[memoryClass];
10813	VMA_ASSERT(innerFreeMap != `0`);
10814	}
10815	// Find lowest free subregion
10816	listIndex = GetListIndex(memoryClass, VMA_BITSCAN_LSB(innerFreeMap));
10817	VMA_ASSERT(m_FreeList[listIndex]);
10818	return m_FreeList[listIndex];
10819	}
10820
10821	bool VmaBlockMetadata_TLSF::CheckBlock(
10822	Block& block,
10823	uint32_t listIndex,
10824	VkDeviceSize allocSize,
10825	VkDeviceSize allocAlignment,
10826	VmaSuballocationType allocType,
10827	VmaAllocationRequest* pAllocationRequest)
10828	{
10829	VMA_ASSERT(block.IsFree() && "Block is already taken!");
10830
10831	VkDeviceSize alignedOffset = VmaAlignUp(block.offset, allocAlignment);
10832	if (block.size < allocSize + alignedOffset - block.offset)
10833	return false;
10834
10835	// Check for granularity conflicts
10836	if (!IsVirtual() &&
10837	m_GranularityHandler.CheckConflictAndAlignUp(alignedOffset, allocSize, block.offset, block.size, allocType))
10838	return false;
10839
10840	// Alloc successful
10841	pAllocationRequest->type = VmaAllocationRequestType::TLSF;
10842	pAllocationRequest->allocHandle = (VmaAllocHandle)&block;
10843	pAllocationRequest->size = allocSize - GetDebugMargin();
10844	pAllocationRequest->customData = (void*)allocType;
10845	pAllocationRequest->algorithmData = alignedOffset;
10846
10847	// Place block at the start of list if it's normal block
10848	if (listIndex != m_ListsCount && block.PrevFree())
10849	{
10850	block.PrevFree()->NextFree() = block.NextFree();
10851	if (block.NextFree())
10852	block.NextFree()->PrevFree() = block.PrevFree();
10853	block.PrevFree() = VMA_NULL;
10854	block.NextFree() = m_FreeList[listIndex];
10855	m_FreeList[listIndex] = &block;
10856	if (block.NextFree())
10857	block.NextFree()->PrevFree() = &block;
10858	}
10859
10860	return true;
10861	}
10862	#endif // _VMA_BLOCK_METADATA_TLSF_FUNCTIONS
10863	#endif // _VMA_BLOCK_METADATA_TLSF
10864
10865	#ifndef _VMA_BLOCK_VECTOR
10866	/*
10867	Sequence of VmaDeviceMemoryBlock. Represents memory blocks allocated for a specific
10868	Vulkan memory type.
10869
10870	Synchronized internally with a mutex.
10871	*/
10872	class VmaBlockVector
10873	{
10874	friend struct VmaDefragmentationContext_T;
10875	VMA_CLASS_NO_COPY(VmaBlockVector)
10876	public:
10877	VmaBlockVector(
10878	VmaAllocator hAllocator,
10879	VmaPool hParentPool,
10880	uint32_t memoryTypeIndex,
10881	VkDeviceSize preferredBlockSize,
10882	size_t minBlockCount,
10883	size_t maxBlockCount,
10884	VkDeviceSize bufferImageGranularity,
10885	bool explicitBlockSize,
10886	uint32_t algorithm,
10887	float priority,
10888	VkDeviceSize minAllocationAlignment,
10889	void* pMemoryAllocateNext);
10890	~VmaBlockVector();
10891
10892	VmaAllocator GetAllocator() const { return m_hAllocator; }
10893	VmaPool GetParentPool() const { return m_hParentPool; }
10894	bool IsCustomPool() const { return m_hParentPool != VMA_NULL; }
10895	uint32_t GetMemoryTypeIndex() const { return m_MemoryTypeIndex; }
10896	VkDeviceSize GetPreferredBlockSize() const { return m_PreferredBlockSize; }
10897	VkDeviceSize GetBufferImageGranularity() const { return m_BufferImageGranularity; }
10898	uint32_t GetAlgorithm() const { return m_Algorithm; }
10899	bool HasExplicitBlockSize() const { return m_ExplicitBlockSize; }
10900	float GetPriority() const { return m_Priority; }
10901	const void* GetAllocationNextPtr() const { return m_pMemoryAllocateNext; }
10902	// To be used only while the m_Mutex is locked. Used during defragmentation.
10903	size_t GetBlockCount() const { return m_Blocks.size(); }
10904	// To be used only while the m_Mutex is locked. Used during defragmentation.
10905	VmaDeviceMemoryBlock* GetBlock(size_t index) const { return m_Blocks[index]; }
10906	VMA_RW_MUTEX &GetMutex() { return m_Mutex; }
10907
10908	VkResult CreateMinBlocks();
10909	void AddStatistics(VmaStatistics& inoutStats);
10910	void AddDetailedStatistics(VmaDetailedStatistics& inoutStats);
10911	bool IsEmpty();
10912	bool IsCorruptionDetectionEnabled() const;
10913
10914	VkResult Allocate(
10915	VkDeviceSize size,
10916	VkDeviceSize alignment,
10917	const VmaAllocationCreateInfo& createInfo,
10918	VmaSuballocationType suballocType,
10919	size_t allocationCount,
10920	VmaAllocation* pAllocations);
10921
10922	void Free(const VmaAllocation hAllocation);
10923
10924	#if VMA_STATS_STRING_ENABLED
10925	void PrintDetailedMap(class VmaJsonWriter& json);
10926	#endif
10927
10928	VkResult CheckCorruption();
10929
10930	private:
10931	const VmaAllocator m_hAllocator;
10932	const VmaPool m_hParentPool;
10933	const uint32_t m_MemoryTypeIndex;
10934	const VkDeviceSize m_PreferredBlockSize;
10935	const size_t m_MinBlockCount;
10936	const size_t m_MaxBlockCount;
10937	const VkDeviceSize m_BufferImageGranularity;
10938	const bool m_ExplicitBlockSize;
10939	const uint32_t m_Algorithm;
10940	const float m_Priority;
10941	const VkDeviceSize m_MinAllocationAlignment;
10942
10943	void* const m_pMemoryAllocateNext;
10944	VMA_RW_MUTEX m_Mutex;
10945	// Incrementally sorted by sumFreeSize, ascending.
10946	VmaVector<VmaDeviceMemoryBlock, VmaStlAllocator<VmaDeviceMemoryBlock>> m_Blocks;
10947	uint32_t m_NextBlockId;
10948	bool m_IncrementalSort = true;
10949
10950	void SetIncrementalSort(bool val) { m_IncrementalSort = val; }
10951
10952	VkDeviceSize CalcMaxBlockSize() const;
10953	// Finds and removes given block from vector.
10954	void Remove(VmaDeviceMemoryBlock* pBlock);
10955	// Performs single step in sorting m_Blocks. They may not be fully sorted
10956	// after this call.
10957	void IncrementallySortBlocks();
10958	void SortByFreeSize();
10959
10960	VkResult AllocatePage(
10961	VkDeviceSize size,
10962	VkDeviceSize alignment,
10963	const VmaAllocationCreateInfo& createInfo,
10964	VmaSuballocationType suballocType,
10965	VmaAllocation* pAllocation);
10966
10967	VkResult AllocateFromBlock(
10968	VmaDeviceMemoryBlock* pBlock,
10969	VkDeviceSize size,
10970	VkDeviceSize alignment,
10971	VmaAllocationCreateFlags allocFlags,
10972	void* pUserData,
10973	VmaSuballocationType suballocType,
10974	uint32_t strategy,
10975	VmaAllocation* pAllocation);
10976
10977	VkResult CommitAllocationRequest(
10978	VmaAllocationRequest& allocRequest,
10979	VmaDeviceMemoryBlock* pBlock,
10980	VkDeviceSize alignment,
10981	VmaAllocationCreateFlags allocFlags,
10982	void* pUserData,
10983	VmaSuballocationType suballocType,
10984	VmaAllocation* pAllocation);
10985
10986	VkResult CreateBlock(VkDeviceSize blockSize, size_t* pNewBlockIndex);
10987	bool HasEmptyBlock();
10988	};
10989	#endif // _VMA_BLOCK_VECTOR
10990
10991	#ifndef _VMA_DEFRAGMENTATION_CONTEXT
10992	struct VmaDefragmentationContext_T
10993	{
10994	VMA_CLASS_NO_COPY(VmaDefragmentationContext_T)
10995	public:
10996	VmaDefragmentationContext_T(
10997	VmaAllocator hAllocator,
10998	const VmaDefragmentationInfo& info);
10999	~VmaDefragmentationContext_T();
11000
11001	void GetStats(VmaDefragmentationStats& outStats) { outStats = m_GlobalStats; }
11002
11003	VkResult DefragmentPassBegin(VmaDefragmentationPassMoveInfo& moveInfo);
11004	VkResult DefragmentPassEnd(VmaDefragmentationPassMoveInfo& moveInfo);
11005
11006	private:
11007	// Max number of allocations to ignore due to size constraints before ending single pass
11008	static const uint8_t MAX_ALLOCS_TO_IGNORE = `16`;
11009	enum class CounterStatus { Pass, Ignore, End };
11010
11011	struct FragmentedBlock
11012	{
11013	uint32_t data;
11014	VmaDeviceMemoryBlock* block;
11015	};
11016	struct StateBalanced
11017	{
11018	VkDeviceSize avgFreeSize = `0`;
11019	VkDeviceSize avgAllocSize = UINT64_MAX;
11020	};
11021	struct StateExtensive
11022	{
11023	enum class Operation : uint8_t
11024	{
11025	FindFreeBlockBuffer, FindFreeBlockTexture, FindFreeBlockAll,
11026	MoveBuffers, MoveTextures, MoveAll,
11027	Cleanup, Done
11028	};
11029
11030	Operation operation = Operation::FindFreeBlockTexture;
11031	size_t firstFreeBlock = SIZE_MAX;
11032	};
11033	struct MoveAllocationData
11034	{
11035	VkDeviceSize size;
11036	VkDeviceSize alignment;
11037	VmaSuballocationType type;
11038	VmaAllocationCreateFlags flags;
11039	VmaDefragmentationMove move = {};
11040	};
11041
11042	const VkDeviceSize m_MaxPassBytes;
11043	const uint32_t m_MaxPassAllocations;
11044
11045	VmaStlAllocator<VmaDefragmentationMove> m_MoveAllocator;
11046	VmaVector<VmaDefragmentationMove, VmaStlAllocator<VmaDefragmentationMove>> m_Moves;
11047
11048	uint8_t m_IgnoredAllocs = `0`;
11049	uint32_t m_Algorithm;
11050	uint32_t m_BlockVectorCount;
11051	VmaBlockVector* m_PoolBlockVector;
11052	VmaBlockVector** m_pBlockVectors;
11053	size_t m_ImmovableBlockCount = `0`;
11054	VmaDefragmentationStats m_GlobalStats = { `0` };
11055	VmaDefragmentationStats m_PassStats = { `0` };
11056	void* m_AlgorithmState = VMA_NULL;
11057
11058	static MoveAllocationData GetMoveData(VmaAllocHandle handle, VmaBlockMetadata* metadata);
11059	CounterStatus CheckCounters(VkDeviceSize bytes);
11060	bool IncrementCounters(VkDeviceSize bytes);
11061	bool ReallocWithinBlock(VmaBlockVector& vector, VmaDeviceMemoryBlock* block);
11062	bool AllocInOtherBlock(size_t start, size_t end, MoveAllocationData& data, VmaBlockVector& vector);
11063
11064	bool ComputeDefragmentation(VmaBlockVector& vector, size_t index);
11065	bool ComputeDefragmentation_Fast(VmaBlockVector& vector);
11066	bool ComputeDefragmentation_Balanced(VmaBlockVector& vector, size_t index, bool update);
11067	bool ComputeDefragmentation_Full(VmaBlockVector& vector);
11068	bool ComputeDefragmentation_Extensive(VmaBlockVector& vector, size_t index);
11069
11070	void UpdateVectorStatistics(VmaBlockVector& vector, StateBalanced& state);
11071	bool MoveDataToFreeBlocks(VmaSuballocationType currentType,
11072	VmaBlockVector& vector, size_t firstFreeBlock,
11073	bool& texturePresent, bool& bufferPresent, bool& otherPresent);
11074	};
11075	#endif // _VMA_DEFRAGMENTATION_CONTEXT
11076
11077	#ifndef _VMA_POOL_T
11078	struct VmaPool_T
11079	{
11080	friend struct VmaPoolListItemTraits;
11081	VMA_CLASS_NO_COPY(VmaPool_T)
11082	public:
11083	VmaBlockVector m_BlockVector;
11084	VmaDedicatedAllocationList m_DedicatedAllocations;
11085
11086	VmaPool_T(
11087	VmaAllocator hAllocator,
11088	const VmaPoolCreateInfo& createInfo,
11089	VkDeviceSize preferredBlockSize);
11090	~VmaPool_T();
11091
11092	uint32_t GetId() const { return m_Id; }
11093	void SetId(uint32_t id) { VMA_ASSERT(m_Id == `0`); m_Id = id; }
11094
11095	const char* GetName() const { return m_Name; }
11096	void SetName(const char* pName);
11097
11098	#if VMA_STATS_STRING_ENABLED
11099	//void PrintDetailedMap(class VmaStringBuilder& sb);
11100	#endif
11101
11102	private:
11103	uint32_t m_Id;
11104	char* m_Name;
11105	VmaPool_T* m_PrevPool = VMA_NULL;
11106	VmaPool_T* m_NextPool = VMA_NULL;
11107	};
11108
11109	struct VmaPoolListItemTraits
11110	{
11111	typedef VmaPool_T ItemType;
11112
11113	static ItemType* GetPrev(const ItemType* item) { return item->m_PrevPool; }
11114	static ItemType* GetNext(const ItemType* item) { return item->m_NextPool; }
11115	static ItemType& AccessPrev(ItemType item) { return item->m_PrevPool; }
11116	static ItemType& AccessNext(ItemType item) { return item->m_NextPool; }
11117	};
11118	#endif // _VMA_POOL_T
11119
11120	#ifndef _VMA_CURRENT_BUDGET_DATA
11121	struct VmaCurrentBudgetData
11122	{
11123	VMA_ATOMIC_UINT32 m_BlockCount[VK_MAX_MEMORY_HEAPS];
11124	VMA_ATOMIC_UINT32 m_AllocationCount[VK_MAX_MEMORY_HEAPS];
11125	VMA_ATOMIC_UINT64 m_BlockBytes[VK_MAX_MEMORY_HEAPS];
11126	VMA_ATOMIC_UINT64 m_AllocationBytes[VK_MAX_MEMORY_HEAPS];
11127
11128	#if VMA_MEMORY_BUDGET
11129	VMA_ATOMIC_UINT32 m_OperationsSinceBudgetFetch;
11130	VMA_RW_MUTEX m_BudgetMutex;
11131	uint64_t m_VulkanUsage[VK_MAX_MEMORY_HEAPS];
11132	uint64_t m_VulkanBudget[VK_MAX_MEMORY_HEAPS];
11133	uint64_t m_BlockBytesAtBudgetFetch[VK_MAX_MEMORY_HEAPS];
11134	#endif // VMA_MEMORY_BUDGET
11135
11136	VmaCurrentBudgetData();
11137
11138	void AddAllocation(uint32_t heapIndex, VkDeviceSize allocationSize);
11139	void RemoveAllocation(uint32_t heapIndex, VkDeviceSize allocationSize);
11140	};
11141
11142	#ifndef _VMA_CURRENT_BUDGET_DATA_FUNCTIONS
11143	VmaCurrentBudgetData::VmaCurrentBudgetData()
11144	{
11145	for (uint32_t heapIndex = `0`; heapIndex < VK_MAX_MEMORY_HEAPS; ++heapIndex)
11146	{
11147	m_BlockCount[heapIndex] = `0`;
11148	m_AllocationCount[heapIndex] = `0`;
11149	m_BlockBytes[heapIndex] = `0`;
11150	m_AllocationBytes[heapIndex] = `0`;
11151	#if VMA_MEMORY_BUDGET
11152	m_VulkanUsage[heapIndex] = `0`;
11153	m_VulkanBudget[heapIndex] = `0`;
11154	m_BlockBytesAtBudgetFetch[heapIndex] = `0`;
11155	#endif
11156	}
11157
11158	#if VMA_MEMORY_BUDGET
11159	m_OperationsSinceBudgetFetch = `0`;
11160	#endif
11161	}
11162
11163	void VmaCurrentBudgetData::AddAllocation(uint32_t heapIndex, VkDeviceSize allocationSize)
11164	{
11165	m_AllocationBytes[heapIndex] += allocationSize;
11166	++m_AllocationCount[heapIndex];
11167	#if VMA_MEMORY_BUDGET
11168	++m_OperationsSinceBudgetFetch;
11169	#endif
11170	}
11171
11172	void VmaCurrentBudgetData::RemoveAllocation(uint32_t heapIndex, VkDeviceSize allocationSize)
11173	{
11174	VMA_ASSERT(m_AllocationBytes[heapIndex] >= allocationSize);
11175	m_AllocationBytes[heapIndex] -= allocationSize;
11176	VMA_ASSERT(m_AllocationCount[heapIndex] > `0`);
11177	--m_AllocationCount[heapIndex];
11178	#if VMA_MEMORY_BUDGET
11179	++m_OperationsSinceBudgetFetch;
11180	#endif
11181	}
11182	#endif // _VMA_CURRENT_BUDGET_DATA_FUNCTIONS
11183	#endif // _VMA_CURRENT_BUDGET_DATA
11184
11185	#ifndef _VMA_ALLOCATION_OBJECT_ALLOCATOR
11186	/*
11187	Thread-safe wrapper over VmaPoolAllocator free list, for allocation of VmaAllocation_T objects.
11188	*/
11189	class VmaAllocationObjectAllocator
11190	{
11191	VMA_CLASS_NO_COPY(VmaAllocationObjectAllocator)
11192	public:
11193	VmaAllocationObjectAllocator(const VkAllocationCallbacks* pAllocationCallbacks)
11194	: m_Allocator(pAllocationCallbacks, `1024`) {}
11195
11196	template<typename... Types> VmaAllocation Allocate(Types&&... args);
11197	void Free(VmaAllocation hAlloc);
11198
11199	private:
11200	VMA_MUTEX m_Mutex;
11201	VmaPoolAllocator<VmaAllocation_T> m_Allocator;
11202	};
11203
11204	template<typename... Types>
11205	VmaAllocation VmaAllocationObjectAllocator::Allocate(Types&&... args)
11206	{
11207	VmaMutexLock mutexLock(m_Mutex);
11208	return m_Allocator.Alloc<Types...>(std::forward<Types>(args)...);
11209	}
11210
11211	void VmaAllocationObjectAllocator::Free(VmaAllocation hAlloc)
11212	{
11213	VmaMutexLock mutexLock(m_Mutex);
11214	m_Allocator.Free(hAlloc);
11215	}
11216	#endif // _VMA_ALLOCATION_OBJECT_ALLOCATOR
11217
11218	#ifndef _VMA_VIRTUAL_BLOCK_T
11219	struct VmaVirtualBlock_T
11220	{
11221	VMA_CLASS_NO_COPY(VmaVirtualBlock_T)
11222	public:
11223	const bool m_AllocationCallbacksSpecified;
11224	const VkAllocationCallbacks m_AllocationCallbacks;
11225
11226	VmaVirtualBlock_T(const VmaVirtualBlockCreateInfo& createInfo);
11227	~VmaVirtualBlock_T();
11228
11229	VkResult Init() { return VK_SUCCESS; }
11230	bool IsEmpty() const { return m_Metadata->IsEmpty(); }
11231	void Free(VmaVirtualAllocation allocation) { m_Metadata->Free((VmaAllocHandle)allocation); }
11232	void SetAllocationUserData(VmaVirtualAllocation allocation, void* userData) { m_Metadata->SetAllocationUserData((VmaAllocHandle)allocation, userData); }
11233	void Clear() { m_Metadata->Clear(); }
11234
11235	const VkAllocationCallbacks* GetAllocationCallbacks() const;
11236	void GetAllocationInfo(VmaVirtualAllocation allocation, VmaVirtualAllocationInfo& outInfo);
11237	VkResult Allocate(const VmaVirtualAllocationCreateInfo& createInfo, VmaVirtualAllocation& outAllocation,
11238	VkDeviceSize* outOffset);
11239	void GetStatistics(VmaStatistics& outStats) const;
11240	void CalculateDetailedStatistics(VmaDetailedStatistics& outStats) const;
11241	#if VMA_STATS_STRING_ENABLED
11242	void BuildStatsString(bool detailedMap, VmaStringBuilder& sb) const;
11243	#endif
11244
11245	private:
11246	VmaBlockMetadata* m_Metadata;
11247	};
11248
11249	#ifndef _VMA_VIRTUAL_BLOCK_T_FUNCTIONS
11250	VmaVirtualBlock_T::VmaVirtualBlock_T(const VmaVirtualBlockCreateInfo& createInfo)
11251	: m_AllocationCallbacksSpecified(createInfo.pAllocationCallbacks != VMA_NULL),
11252	m_AllocationCallbacks(createInfo.pAllocationCallbacks != VMA_NULL ? *createInfo.pAllocationCallbacks : VmaEmptyAllocationCallbacks)
11253	{
11254	const uint32_t algorithm = createInfo.flags & VMA_VIRTUAL_BLOCK_CREATE_ALGORITHM_MASK;
11255	switch (algorithm)
11256	{
11257	default:
11258	VMA_ASSERT(`0`);
11259	case `0`:
11260	m_Metadata = vma_new(GetAllocationCallbacks(), VmaBlockMetadata_TLSF)(VK_NULL_HANDLE, `1`, true);
11261	break;
11262	case VMA_VIRTUAL_BLOCK_CREATE_LINEAR_ALGORITHM_BIT:
11263	m_Metadata = vma_new(GetAllocationCallbacks(), VmaBlockMetadata_Linear)(VK_NULL_HANDLE, `1`, true);
11264	break;
11265	}
11266
11267	m_Metadata->Init(createInfo.size);
11268	}
11269
11270	VmaVirtualBlock_T::~VmaVirtualBlock_T()
11271	{
11272	// Define macro VMA_DEBUG_LOG to receive the list of the unfreed allocations
11273	if (!m_Metadata->IsEmpty())
11274	m_Metadata->DebugLogAllAllocations();
11275	// This is the most important assert in the entire library.
11276	// Hitting it means you have some memory leak - unreleased virtual allocations.
11277	VMA_ASSERT(m_Metadata->IsEmpty() && "Some virtual allocations were not freed before destruction of this virtual block!");
11278
11279	vma_delete(GetAllocationCallbacks(), m_Metadata);
11280	}
11281
11282	const VkAllocationCallbacks* VmaVirtualBlock_T::GetAllocationCallbacks() const
11283	{
11284	return m_AllocationCallbacksSpecified ? &m_AllocationCallbacks : VMA_NULL;
11285	}
11286
11287	void VmaVirtualBlock_T::GetAllocationInfo(VmaVirtualAllocation allocation, VmaVirtualAllocationInfo& outInfo)
11288	{
11289	m_Metadata->GetAllocationInfo((VmaAllocHandle)allocation, outInfo);
11290	}
11291
11292	VkResult VmaVirtualBlock_T::Allocate(const VmaVirtualAllocationCreateInfo& createInfo, VmaVirtualAllocation& outAllocation,
11293	VkDeviceSize* outOffset)
11294	{
11295	VmaAllocationRequest request = {};
11296	if (m_Metadata->CreateAllocationRequest(
11297	createInfo.size, // allocSize
11298	VMA_MAX(createInfo.alignment, (VkDeviceSize)`1`), // allocAlignment
11299	(createInfo.flags & VMA_VIRTUAL_ALLOCATION_CREATE_UPPER_ADDRESS_BIT) != `0`, // upperAddress
11300	VMA_SUBALLOCATION_TYPE_UNKNOWN, // allocType - unimportant
11301	createInfo.flags & VMA_VIRTUAL_ALLOCATION_CREATE_STRATEGY_MASK, // strategy
11302	&request))
11303	{
11304	m_Metadata->Alloc(request,
11305	VMA_SUBALLOCATION_TYPE_UNKNOWN, // type - unimportant
11306	createInfo.pUserData);
11307	outAllocation = (VmaVirtualAllocation)request.allocHandle;
11308	if(outOffset)
11309	*outOffset = m_Metadata->GetAllocationOffset(request.allocHandle);
11310	return VK_SUCCESS;
11311	}
11312	outAllocation = (VmaVirtualAllocation)VK_NULL_HANDLE;
11313	if (outOffset)
11314	*outOffset = UINT64_MAX;
11315	return VK_ERROR_OUT_OF_DEVICE_MEMORY;
11316	}
11317
11318	void VmaVirtualBlock_T::GetStatistics(VmaStatistics& outStats) const
11319	{
11320	VmaClearStatistics(outStats);
11321	m_Metadata->AddStatistics(outStats);
11322	}
11323
11324	void VmaVirtualBlock_T::CalculateDetailedStatistics(VmaDetailedStatistics& outStats) const
11325	{
11326	VmaClearDetailedStatistics(outStats);
11327	m_Metadata->AddDetailedStatistics(outStats);
11328	}
11329
11330	#if VMA_STATS_STRING_ENABLED
11331	void VmaVirtualBlock_T::BuildStatsString(bool detailedMap, VmaStringBuilder& sb) const
11332	{
11333	VmaJsonWriter json(GetAllocationCallbacks(), sb);
11334	json.BeginObject();
11335
11336	VmaDetailedStatistics stats;
11337	CalculateDetailedStatistics(stats);
11338
11339	json.WriteString("Stats");
11340	VmaPrintDetailedStatistics(json, stats);
11341
11342	if (detailedMap)
11343	{
11344	json.WriteString("Details");
11345	json.BeginObject();
11346	m_Metadata->PrintDetailedMap(json);
11347	json.EndObject();
11348	}
11349
11350	json.EndObject();
11351	}
11352	#endif // VMA_STATS_STRING_ENABLED
11353	#endif // _VMA_VIRTUAL_BLOCK_T_FUNCTIONS
11354	#endif // _VMA_VIRTUAL_BLOCK_T
11355
11356
11357	// Main allocator object.
11358	struct VmaAllocator_T
11359	{
11360	VMA_CLASS_NO_COPY(VmaAllocator_T)
11361	public:
11362	bool m_UseMutex;
11363	uint32_t m_VulkanApiVersion;
11364	bool m_UseKhrDedicatedAllocation; // Can be set only if m_VulkanApiVersion < VK_MAKE_VERSION(1, 1, 0).
11365	bool m_UseKhrBindMemory2; // Can be set only if m_VulkanApiVersion < VK_MAKE_VERSION(1, 1, 0).
11366	bool m_UseExtMemoryBudget;
11367	bool m_UseAmdDeviceCoherentMemory;
11368	bool m_UseKhrBufferDeviceAddress;
11369	bool m_UseExtMemoryPriority;
11370	VkDevice m_hDevice;
11371	VkInstance m_hInstance;
11372	bool m_AllocationCallbacksSpecified;
11373	VkAllocationCallbacks m_AllocationCallbacks;
11374	VmaDeviceMemoryCallbacks m_DeviceMemoryCallbacks;
11375	VmaAllocationObjectAllocator m_AllocationObjectAllocator;
11376
11377	// Each bit (1 << i) is set if HeapSizeLimit is enabled for that heap, so cannot allocate more than the heap size.
11378	uint32_t m_HeapSizeLimitMask;
11379
11380	VkPhysicalDeviceProperties m_PhysicalDeviceProperties;
11381	VkPhysicalDeviceMemoryProperties m_MemProps;
11382
11383	// Default pools.
11384	VmaBlockVector* m_pBlockVectors[VK_MAX_MEMORY_TYPES];
11385	VmaDedicatedAllocationList m_DedicatedAllocations[VK_MAX_MEMORY_TYPES];
11386
11387	VmaCurrentBudgetData m_Budget;
11388	VMA_ATOMIC_UINT32 m_DeviceMemoryCount; // Total number of VkDeviceMemory objects.
11389
11390	VmaAllocator_T(const VmaAllocatorCreateInfo* pCreateInfo);
11391	VkResult Init(const VmaAllocatorCreateInfo* pCreateInfo);
11392	~VmaAllocator_T();
11393
11394	const VkAllocationCallbacks* GetAllocationCallbacks() const
11395	{
11396	return m_AllocationCallbacksSpecified ? &m_AllocationCallbacks : VMA_NULL;
11397	}
11398	const VmaVulkanFunctions& GetVulkanFunctions() const
11399	{
11400	return m_VulkanFunctions;
11401	}
11402
11403	VkPhysicalDevice GetPhysicalDevice() const { return m_PhysicalDevice; }
11404
11405	VkDeviceSize GetBufferImageGranularity() const
11406	{
11407	return VMA_MAX(
11408	static_cast<VkDeviceSize>(VMA_DEBUG_MIN_BUFFER_IMAGE_GRANULARITY),
11409	m_PhysicalDeviceProperties.limits.bufferImageGranularity);
11410	}
11411
11412	uint32_t GetMemoryHeapCount() const { return m_MemProps.memoryHeapCount; }
11413	uint32_t GetMemoryTypeCount() const { return m_MemProps.memoryTypeCount; }
11414
11415	uint32_t MemoryTypeIndexToHeapIndex(uint32_t memTypeIndex) const
11416	{
11417	VMA_ASSERT(memTypeIndex < m_MemProps.memoryTypeCount);
11418	return m_MemProps.memoryTypes[memTypeIndex].heapIndex;
11419	}
11420	// True when specific memory type is HOST_VISIBLE but not HOST_COHERENT.
11421	bool IsMemoryTypeNonCoherent(uint32_t memTypeIndex) const
11422	{
11423	return (m_MemProps.memoryTypes[memTypeIndex].propertyFlags & (VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT \| VK_MEMORY_PROPERTY_HOST_COHERENT_BIT)) ==
11424	VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT;
11425	}
11426	// Minimum alignment for all allocations in specific memory type.
11427	VkDeviceSize GetMemoryTypeMinAlignment(uint32_t memTypeIndex) const
11428	{
11429	return IsMemoryTypeNonCoherent(memTypeIndex) ?
11430	VMA_MAX((VkDeviceSize)VMA_MIN_ALIGNMENT, m_PhysicalDeviceProperties.limits.nonCoherentAtomSize) :
11431	(VkDeviceSize)VMA_MIN_ALIGNMENT;
11432	}
11433
11434	bool IsIntegratedGpu() const
11435	{
11436	return m_PhysicalDeviceProperties.deviceType == VK_PHYSICAL_DEVICE_TYPE_INTEGRATED_GPU;
11437	}
11438
11439	uint32_t GetGlobalMemoryTypeBits() const { return m_GlobalMemoryTypeBits; }
11440
11441	void GetBufferMemoryRequirements(
11442	VkBuffer hBuffer,
11443	VkMemoryRequirements& memReq,
11444	bool& requiresDedicatedAllocation,
11445	bool& prefersDedicatedAllocation) const;
11446	void GetImageMemoryRequirements(
11447	VkImage hImage,
11448	VkMemoryRequirements& memReq,
11449	bool& requiresDedicatedAllocation,
11450	bool& prefersDedicatedAllocation) const;
11451	VkResult FindMemoryTypeIndex(
11452	uint32_t memoryTypeBits,
11453	const VmaAllocationCreateInfo* pAllocationCreateInfo,
11454	VkFlags bufImgUsage, // VkBufferCreateInfo::usage or VkImageCreateInfo::usage. UINT32_MAX if unknown.
11455	uint32_t* pMemoryTypeIndex) const;
11456
11457	// Main allocation function.
11458	VkResult AllocateMemory(
11459	const VkMemoryRequirements& vkMemReq,
11460	bool requiresDedicatedAllocation,
11461	bool prefersDedicatedAllocation,
11462	VkBuffer dedicatedBuffer,
11463	VkImage dedicatedImage,
11464	VkFlags dedicatedBufferImageUsage, // UINT32_MAX if unknown.
11465	const VmaAllocationCreateInfo& createInfo,
11466	VmaSuballocationType suballocType,
11467	size_t allocationCount,
11468	VmaAllocation* pAllocations);
11469
11470	// Main deallocation function.
11471	void FreeMemory(
11472	size_t allocationCount,
11473	const VmaAllocation* pAllocations);
11474
11475	void CalculateStatistics(VmaTotalStatistics* pStats);
11476
11477	void GetHeapBudgets(
11478	VmaBudget* outBudgets, uint32_t firstHeap, uint32_t heapCount);
11479
11480	#if VMA_STATS_STRING_ENABLED
11481	void PrintDetailedMap(class VmaJsonWriter& json);
11482	#endif
11483
11484	void GetAllocationInfo(VmaAllocation hAllocation, VmaAllocationInfo* pAllocationInfo);
11485
11486	VkResult CreatePool(const VmaPoolCreateInfo* pCreateInfo, VmaPool* pPool);
11487	void DestroyPool(VmaPool pool);
11488	void GetPoolStatistics(VmaPool pool, VmaStatistics* pPoolStats);
11489	void CalculatePoolStatistics(VmaPool pool, VmaDetailedStatistics* pPoolStats);
11490
11491	void SetCurrentFrameIndex(uint32_t frameIndex);
11492	uint32_t GetCurrentFrameIndex() const { return m_CurrentFrameIndex.load(); }
11493
11494	VkResult CheckPoolCorruption(VmaPool hPool);
11495	VkResult CheckCorruption(uint32_t memoryTypeBits);
11496
11497	// Call to Vulkan function vkAllocateMemory with accompanying bookkeeping.
11498	VkResult AllocateVulkanMemory(const VkMemoryAllocateInfo* pAllocateInfo, VkDeviceMemory* pMemory);
11499	// Call to Vulkan function vkFreeMemory with accompanying bookkeeping.
11500	void FreeVulkanMemory(uint32_t memoryType, VkDeviceSize size, VkDeviceMemory hMemory);
11501	// Call to Vulkan function vkBindBufferMemory or vkBindBufferMemory2KHR.
11502	VkResult BindVulkanBuffer(
11503	VkDeviceMemory memory,
11504	VkDeviceSize memoryOffset,
11505	VkBuffer buffer,
11506	const void* pNext);
11507	// Call to Vulkan function vkBindImageMemory or vkBindImageMemory2KHR.
11508	VkResult BindVulkanImage(
11509	VkDeviceMemory memory,
11510	VkDeviceSize memoryOffset,
11511	VkImage image,
11512	const void* pNext);
11513
11514	VkResult Map(VmaAllocation hAllocation, void** ppData);
11515	void Unmap(VmaAllocation hAllocation);
11516
11517	VkResult BindBufferMemory(
11518	VmaAllocation hAllocation,
11519	VkDeviceSize allocationLocalOffset,
11520	VkBuffer hBuffer,
11521	const void* pNext);
11522	VkResult BindImageMemory(
11523	VmaAllocation hAllocation,
11524	VkDeviceSize allocationLocalOffset,
11525	VkImage hImage,
11526	const void* pNext);
11527
11528	VkResult FlushOrInvalidateAllocation(
11529	VmaAllocation hAllocation,
11530	VkDeviceSize offset, VkDeviceSize size,
11531	VMA_CACHE_OPERATION op);
11532	VkResult FlushOrInvalidateAllocations(
11533	uint32_t allocationCount,
11534	const VmaAllocation* allocations,
11535	const VkDeviceSize* offsets, const VkDeviceSize* sizes,
11536	VMA_CACHE_OPERATION op);
11537
11538	void FillAllocation(const VmaAllocation hAllocation, uint8_t pattern);
11539
11540	/*
11541	Returns bit mask of memory types that can support defragmentation on GPU as
11542	they support creation of required buffer for copy operations.
11543	*/
11544	uint32_t GetGpuDefragmentationMemoryTypeBits();
11545
11546	#if VMA_EXTERNAL_MEMORY
11547	VkExternalMemoryHandleTypeFlagsKHR GetExternalMemoryHandleTypeFlags(uint32_t memTypeIndex) const
11548	{
11549	return m_TypeExternalMemoryHandleTypes[memTypeIndex];
11550	}
11551	#endif // #if VMA_EXTERNAL_MEMORY
11552
11553	private:
11554	VkDeviceSize m_PreferredLargeHeapBlockSize;
11555
11556	VkPhysicalDevice m_PhysicalDevice;
11557	VMA_ATOMIC_UINT32 m_CurrentFrameIndex;
11558	VMA_ATOMIC_UINT32 m_GpuDefragmentationMemoryTypeBits; // UINT32_MAX means uninitialized.
11559	#if VMA_EXTERNAL_MEMORY
11560	VkExternalMemoryHandleTypeFlagsKHR m_TypeExternalMemoryHandleTypes[VK_MAX_MEMORY_TYPES];
11561	#endif // #if VMA_EXTERNAL_MEMORY
11562
11563	VMA_RW_MUTEX m_PoolsMutex;
11564	typedef VmaIntrusiveLinkedList<VmaPoolListItemTraits> PoolList;
11565	// Protected by m_PoolsMutex.
11566	PoolList m_Pools;
11567	uint32_t m_NextPoolId;
11568
11569	VmaVulkanFunctions m_VulkanFunctions;
11570
11571	// Global bit mask AND-ed with any memoryTypeBits to disallow certain memory types.
11572	uint32_t m_GlobalMemoryTypeBits;
11573
11574	void ImportVulkanFunctions(const VmaVulkanFunctions* pVulkanFunctions);
11575
11576	#if VMA_STATIC_VULKAN_FUNCTIONS == 1
11577	void ImportVulkanFunctions_Static();
11578	#endif
11579
11580	void ImportVulkanFunctions_Custom(const VmaVulkanFunctions* pVulkanFunctions);
11581
11582	#if VMA_DYNAMIC_VULKAN_FUNCTIONS == 1
11583	void ImportVulkanFunctions_Dynamic();
11584	#endif
11585
11586	void ValidateVulkanFunctions();
11587
11588	VkDeviceSize CalcPreferredBlockSize(uint32_t memTypeIndex);
11589
11590	VkResult AllocateMemoryOfType(
11591	VmaPool pool,
11592	VkDeviceSize size,
11593	VkDeviceSize alignment,
11594	bool dedicatedPreferred,
11595	VkBuffer dedicatedBuffer,
11596	VkImage dedicatedImage,
11597	VkFlags dedicatedBufferImageUsage,
11598	const VmaAllocationCreateInfo& createInfo,
11599	uint32_t memTypeIndex,
11600	VmaSuballocationType suballocType,
11601	VmaDedicatedAllocationList& dedicatedAllocations,
11602	VmaBlockVector& blockVector,
11603	size_t allocationCount,
11604	VmaAllocation* pAllocations);
11605
11606	// Helper function only to be used inside AllocateDedicatedMemory.
11607	VkResult AllocateDedicatedMemoryPage(
11608	VmaPool pool,
11609	VkDeviceSize size,
11610	VmaSuballocationType suballocType,
11611	uint32_t memTypeIndex,
11612	const VkMemoryAllocateInfo& allocInfo,
11613	bool map,
11614	bool isUserDataString,
11615	bool isMappingAllowed,
11616	void* pUserData,
11617	VmaAllocation* pAllocation);
11618
11619	// Allocates and registers new VkDeviceMemory specifically for dedicated allocations.
11620	VkResult AllocateDedicatedMemory(
11621	VmaPool pool,
11622	VkDeviceSize size,
11623	VmaSuballocationType suballocType,
11624	VmaDedicatedAllocationList& dedicatedAllocations,
11625	uint32_t memTypeIndex,
11626	bool map,
11627	bool isUserDataString,
11628	bool isMappingAllowed,
11629	bool canAliasMemory,
11630	void* pUserData,
11631	float priority,
11632	VkBuffer dedicatedBuffer,
11633	VkImage dedicatedImage,
11634	VkFlags dedicatedBufferImageUsage,
11635	size_t allocationCount,
11636	VmaAllocation* pAllocations,
11637	const void* pNextChain = nullptr);
11638
11639	void FreeDedicatedMemory(const VmaAllocation allocation);
11640
11641	VkResult CalcMemTypeParams(
11642	VmaAllocationCreateInfo& outCreateInfo,
11643	uint32_t memTypeIndex,
11644	VkDeviceSize size,
11645	size_t allocationCount);
11646	VkResult CalcAllocationParams(
11647	VmaAllocationCreateInfo& outCreateInfo,
11648	bool dedicatedRequired,
11649	bool dedicatedPreferred);
11650
11651	/*
11652	Calculates and returns bit mask of memory types that can support defragmentation
11653	on GPU as they support creation of required buffer for copy operations.
11654	*/
11655	uint32_t CalculateGpuDefragmentationMemoryTypeBits() const;
11656	uint32_t CalculateGlobalMemoryTypeBits() const;
11657
11658	bool GetFlushOrInvalidateRange(
11659	VmaAllocation allocation,
11660	VkDeviceSize offset, VkDeviceSize size,
11661	VkMappedMemoryRange& outRange) const;
11662
11663	#if VMA_MEMORY_BUDGET
11664	void UpdateVulkanBudget();
11665	#endif // #if VMA_MEMORY_BUDGET
11666	};
11667
11668
11669	#ifndef _VMA_MEMORY_FUNCTIONS
11670	static void* VmaMalloc(VmaAllocator hAllocator, size_t size, size_t alignment)
11671	{
11672	return VmaMalloc(&hAllocator->m_AllocationCallbacks, size, alignment);
11673	}
11674
11675	static void VmaFree(VmaAllocator hAllocator, void* ptr)
11676	{
11677	VmaFree(&hAllocator->m_AllocationCallbacks, ptr);
11678	}
11679
11680	template<typename T>
11681	static T* VmaAllocate(VmaAllocator hAllocator)
11682	{
11683	return (T)VmaMalloc(hAllocator, sizeof*(T), VMA_ALIGN_OF(T));
11684	}
11685
11686	template<typename T>
11687	static T* VmaAllocateArray(VmaAllocator hAllocator, size_t count)
11688	{
11689	return (T)VmaMalloc(hAllocator, sizeof(T) count, VMA_ALIGN_OF(T));
11690	}
11691
11692	template<typename T>
11693	static void vma_delete(VmaAllocator hAllocator, T* ptr)
11694	{
11695	if(ptr != VMA_NULL)
11696	{
11697	ptr->~T();
11698	VmaFree(hAllocator, ptr);
11699	}
11700	}
11701
11702	template<typename T>
11703	static void vma_delete_array(VmaAllocator hAllocator, T* ptr, size_t count)
11704	{
11705	if(ptr != VMA_NULL)
11706	{
11707	for(size_t i = count; i--; )
11708	ptr[i].~T();
11709	VmaFree(hAllocator, ptr);
11710	}
11711	}
11712	#endif // _VMA_MEMORY_FUNCTIONS
11713
11714	#ifndef _VMA_DEVICE_MEMORY_BLOCK_FUNCTIONS
11715	VmaDeviceMemoryBlock::VmaDeviceMemoryBlock(VmaAllocator hAllocator)
11716	: m_pMetadata(VMA_NULL),
11717	m_MemoryTypeIndex(UINT32_MAX),
11718	m_Id(`0`),
11719	m_hMemory(VK_NULL_HANDLE),
11720	m_MapCount(`0`),
11721	m_pMappedData(VMA_NULL) {}
11722
11723	VmaDeviceMemoryBlock::~VmaDeviceMemoryBlock()
11724	{
11725	VMA_ASSERT(m_MapCount == `0` && "VkDeviceMemory block is being destroyed while it is still mapped.");
11726	VMA_ASSERT(m_hMemory == VK_NULL_HANDLE);
11727	}
11728
11729	void VmaDeviceMemoryBlock::Init(
11730	VmaAllocator hAllocator,
11731	VmaPool hParentPool,
11732	uint32_t newMemoryTypeIndex,
11733	VkDeviceMemory newMemory,
11734	VkDeviceSize newSize,
11735	uint32_t id,
11736	uint32_t algorithm,
11737	VkDeviceSize bufferImageGranularity)
11738	{
11739	VMA_ASSERT(m_hMemory == VK_NULL_HANDLE);
11740
11741	m_hParentPool = hParentPool;
11742	m_MemoryTypeIndex = newMemoryTypeIndex;
11743	m_Id = id;
11744	m_hMemory = newMemory;
11745
11746	switch (algorithm)
11747	{
11748	case VMA_POOL_CREATE_LINEAR_ALGORITHM_BIT:
11749	m_pMetadata = vma_new(hAllocator, VmaBlockMetadata_Linear)(hAllocator->GetAllocationCallbacks(),
11750	bufferImageGranularity, false); // isVirtual
11751	break;
11752	default:
11753	VMA_ASSERT(`0`);
11754	// Fall-through.
11755	case `0`:
11756	m_pMetadata = vma_new(hAllocator, VmaBlockMetadata_TLSF)(hAllocator->GetAllocationCallbacks(),
11757	bufferImageGranularity, false); // isVirtual
11758	}
11759	m_pMetadata->Init(newSize);
11760	}
11761
11762	void VmaDeviceMemoryBlock::Destroy(VmaAllocator allocator)
11763	{
11764	// Define macro VMA_DEBUG_LOG to receive the list of the unfreed allocations
11765	if (!m_pMetadata->IsEmpty())
11766	m_pMetadata->DebugLogAllAllocations();
11767	// This is the most important assert in the entire library.
11768	// Hitting it means you have some memory leak - unreleased VmaAllocation objects.
11769	VMA_ASSERT(m_pMetadata->IsEmpty() && "Some allocations were not freed before destruction of this memory block!");
11770
11771	VMA_ASSERT(m_hMemory != VK_NULL_HANDLE);
11772	allocator->FreeVulkanMemory(m_MemoryTypeIndex, m_pMetadata->GetSize(), m_hMemory);
11773	m_hMemory = VK_NULL_HANDLE;
11774
11775	vma_delete(allocator, m_pMetadata);
11776	m_pMetadata = VMA_NULL;
11777	}
11778
11779	void VmaDeviceMemoryBlock::PostFree(VmaAllocator hAllocator)
11780	{
11781	if(m_MappingHysteresis.PostFree())
11782	{
11783	VMA_ASSERT(m_MappingHysteresis.GetExtraMapping() == `0`);
11784	if (m_MapCount == `0`)
11785	{
11786	m_pMappedData = VMA_NULL;
11787	(*hAllocator->GetVulkanFunctions().vkUnmapMemory)(hAllocator->m_hDevice, m_hMemory);
11788	}
11789	}
11790	}
11791
11792	bool VmaDeviceMemoryBlock::Validate() const
11793	{
11794	VMA_VALIDATE((m_hMemory != VK_NULL_HANDLE) &&
11795	(m_pMetadata->GetSize() != `0`));
11796
11797	return m_pMetadata->Validate();
11798	}
11799
11800	VkResult VmaDeviceMemoryBlock::CheckCorruption(VmaAllocator hAllocator)
11801	{
11802	void* pData = nullptr;
11803	VkResult res = Map(hAllocator, `1`, &pData);
11804	if (res != VK_SUCCESS)
11805	{
11806	return res;
11807	}
11808
11809	res = m_pMetadata->CheckCorruption(pData);
11810
11811	Unmap(hAllocator, `1`);
11812
11813	return res;
11814	}
11815
11816	VkResult VmaDeviceMemoryBlock::Map(VmaAllocator hAllocator, uint32_t count, void** ppData)
11817	{
11818	if (count == `0`)
11819	{
11820	return VK_SUCCESS;
11821	}
11822
11823	VmaMutexLock lock(m_MapAndBindMutex, hAllocator->m_UseMutex);
11824	const uint32_t oldTotalMapCount = m_MapCount + m_MappingHysteresis.GetExtraMapping();
11825	m_MappingHysteresis.PostMap();
11826	if (oldTotalMapCount != `0`)
11827	{
11828	m_MapCount += count;
11829	VMA_ASSERT(m_pMappedData != VMA_NULL);
11830	if (ppData != VMA_NULL)
11831	{
11832	*ppData = m_pMappedData;
11833	}
11834	return VK_SUCCESS;
11835	}
11836	else
11837	{
11838	VkResult result = (*hAllocator->GetVulkanFunctions().vkMapMemory)(
11839	hAllocator->m_hDevice,
11840	m_hMemory,
11841	`0`, // offset
11842	VK_WHOLE_SIZE,
11843	`0`, // flags
11844	&m_pMappedData);
11845	if (result == VK_SUCCESS)
11846	{
11847	if (ppData != VMA_NULL)
11848	{
11849	*ppData = m_pMappedData;
11850	}
11851	m_MapCount = count;
11852	}
11853	return result;
11854	}
11855	}
11856
11857	void VmaDeviceMemoryBlock::Unmap(VmaAllocator hAllocator, uint32_t count)
11858	{
11859	if (count == `0`)
11860	{
11861	return;
11862	}
11863
11864	VmaMutexLock lock(m_MapAndBindMutex, hAllocator->m_UseMutex);
11865	if (m_MapCount >= count)
11866	{
11867	m_MapCount -= count;
11868	const uint32_t totalMapCount = m_MapCount + m_MappingHysteresis.GetExtraMapping();
11869	if (totalMapCount == `0`)
11870	{
11871	m_pMappedData = VMA_NULL;
11872	(*hAllocator->GetVulkanFunctions().vkUnmapMemory)(hAllocator->m_hDevice, m_hMemory);
11873	}
11874	m_MappingHysteresis.PostUnmap();
11875	}
11876	else
11877	{
11878	VMA_ASSERT(`0` && "VkDeviceMemory block is being unmapped while it was not previously mapped.");
11879	}
11880	}
11881
11882	VkResult VmaDeviceMemoryBlock::WriteMagicValueAfterAllocation(VmaAllocator hAllocator, VkDeviceSize allocOffset, VkDeviceSize allocSize)
11883	{
11884	VMA_ASSERT(VMA_DEBUG_MARGIN > `0` && VMA_DEBUG_MARGIN % `4` == `0` && VMA_DEBUG_DETECT_CORRUPTION);
11885
11886	void* pData;
11887	VkResult res = Map(hAllocator, `1`, &pData);
11888	if (res != VK_SUCCESS)
11889	{
11890	return res;
11891	}
11892
11893	VmaWriteMagicValue(pData, allocOffset + allocSize);
11894
11895	Unmap(hAllocator, `1`);
11896	return VK_SUCCESS;
11897	}
11898
11899	VkResult VmaDeviceMemoryBlock::ValidateMagicValueAfterAllocation(VmaAllocator hAllocator, VkDeviceSize allocOffset, VkDeviceSize allocSize)
11900	{
11901	VMA_ASSERT(VMA_DEBUG_MARGIN > `0` && VMA_DEBUG_MARGIN % `4` == `0` && VMA_DEBUG_DETECT_CORRUPTION);
11902
11903	void* pData;
11904	VkResult res = Map(hAllocator, `1`, &pData);
11905	if (res != VK_SUCCESS)
11906	{
11907	return res;
11908	}
11909
11910	if (!VmaValidateMagicValue(pData, allocOffset + allocSize))
11911	{
11912	VMA_ASSERT(`0` && "MEMORY CORRUPTION DETECTED AFTER FREED ALLOCATION!");
11913	}
11914
11915	Unmap(hAllocator, `1`);
11916	return VK_SUCCESS;
11917	}
11918
11919	VkResult VmaDeviceMemoryBlock::BindBufferMemory(
11920	const VmaAllocator hAllocator,
11921	const VmaAllocation hAllocation,
11922	VkDeviceSize allocationLocalOffset,
11923	VkBuffer hBuffer,
11924	const void* pNext)
11925	{
11926	VMA_ASSERT(hAllocation->GetType() == VmaAllocation_T::ALLOCATION_TYPE_BLOCK &&
11927	hAllocation->GetBlock() == this);
11928	VMA_ASSERT(allocationLocalOffset < hAllocation->GetSize() &&
11929	"Invalid allocationLocalOffset. Did you forget that this offset is relative to the beginning of the allocation, not the whole memory block?");
11930	const VkDeviceSize memoryOffset = hAllocation->GetOffset() + allocationLocalOffset;
11931	// This lock is important so that we don't call vkBind... and/or vkMap... simultaneously on the same VkDeviceMemory from multiple threads.
11932	VmaMutexLock lock(m_MapAndBindMutex, hAllocator->m_UseMutex);
11933	return hAllocator->BindVulkanBuffer(m_hMemory, memoryOffset, hBuffer, pNext);
11934	}
11935
11936	VkResult VmaDeviceMemoryBlock::BindImageMemory(
11937	const VmaAllocator hAllocator,
11938	const VmaAllocation hAllocation,
11939	VkDeviceSize allocationLocalOffset,
11940	VkImage hImage,
11941	const void* pNext)
11942	{
11943	VMA_ASSERT(hAllocation->GetType() == VmaAllocation_T::ALLOCATION_TYPE_BLOCK &&
11944	hAllocation->GetBlock() == this);
11945	VMA_ASSERT(allocationLocalOffset < hAllocation->GetSize() &&
11946	"Invalid allocationLocalOffset. Did you forget that this offset is relative to the beginning of the allocation, not the whole memory block?");
11947	const VkDeviceSize memoryOffset = hAllocation->GetOffset() + allocationLocalOffset;
11948	// This lock is important so that we don't call vkBind... and/or vkMap... simultaneously on the same VkDeviceMemory from multiple threads.
11949	VmaMutexLock lock(m_MapAndBindMutex, hAllocator->m_UseMutex);
11950	return hAllocator->BindVulkanImage(m_hMemory, memoryOffset, hImage, pNext);
11951	}
11952	#endif // _VMA_DEVICE_MEMORY_BLOCK_FUNCTIONS
11953
11954	#ifndef _VMA_ALLOCATION_T_FUNCTIONS
11955	VmaAllocation_T::VmaAllocation_T(bool mappingAllowed)
11956	: m_Alignment{ `1` },
11957	m_Size{ `0` },
11958	m_pUserData{ VMA_NULL },
11959	m_pName{ VMA_NULL },
11960	m_MemoryTypeIndex{ `0` },
11961	m_Type{ (uint8_t)ALLOCATION_TYPE_NONE },
11962	m_SuballocationType{ (uint8_t)VMA_SUBALLOCATION_TYPE_UNKNOWN },
11963	m_MapCount{ `0` },
11964	m_Flags{ `0` }
11965	{
11966	if(mappingAllowed)
11967	m_Flags \|= (uint8_t)FLAG_MAPPING_ALLOWED;
11968
11969	#if VMA_STATS_STRING_ENABLED
11970	m_BufferImageUsage = `0`;
11971	#endif
11972	}
11973
11974	VmaAllocation_T::~VmaAllocation_T()
11975	{
11976	VMA_ASSERT(m_MapCount == `0` && "Allocation was not unmapped before destruction.");
11977
11978	// Check if owned string was freed.
11979	VMA_ASSERT(m_pName == VMA_NULL);
11980	}
11981
11982	void VmaAllocation_T::InitBlockAllocation(
11983	VmaDeviceMemoryBlock* block,
11984	VmaAllocHandle allocHandle,
11985	VkDeviceSize alignment,
11986	VkDeviceSize size,
11987	uint32_t memoryTypeIndex,
11988	VmaSuballocationType suballocationType,
11989	bool mapped)
11990	{
11991	VMA_ASSERT(m_Type == ALLOCATION_TYPE_NONE);
11992	VMA_ASSERT(block != VMA_NULL);
11993	m_Type = (uint8_t)ALLOCATION_TYPE_BLOCK;
11994	m_Alignment = alignment;
11995	m_Size = size;
11996	m_MemoryTypeIndex = memoryTypeIndex;
11997	if(mapped)
11998	{
11999	VMA_ASSERT(IsMappingAllowed() && "Mapping is not allowed on this allocation! Please use one of the new VMA_ALLOCATION_CREATE_HOST_ACCESS_* flags when creating it.");
12000	m_Flags \|= (uint8_t)FLAG_PERSISTENT_MAP;
12001	}
12002	m_SuballocationType = (uint8_t)suballocationType;
12003	m_BlockAllocation.m_Block = block;
12004	m_BlockAllocation.m_AllocHandle = allocHandle;
12005	}
12006
12007	void VmaAllocation_T::InitDedicatedAllocation(
12008	VmaPool hParentPool,
12009	uint32_t memoryTypeIndex,
12010	VkDeviceMemory hMemory,
12011	VmaSuballocationType suballocationType,
12012	void* pMappedData,
12013	VkDeviceSize size)
12014	{
12015	VMA_ASSERT(m_Type == ALLOCATION_TYPE_NONE);
12016	VMA_ASSERT(hMemory != VK_NULL_HANDLE);
12017	m_Type = (uint8_t)ALLOCATION_TYPE_DEDICATED;
12018	m_Alignment = `0`;
12019	m_Size = size;
12020	m_MemoryTypeIndex = memoryTypeIndex;
12021	m_SuballocationType = (uint8_t)suballocationType;
12022	if(pMappedData != VMA_NULL)
12023	{
12024	VMA_ASSERT(IsMappingAllowed() && "Mapping is not allowed on this allocation! Please use one of the new VMA_ALLOCATION_CREATE_HOST_ACCESS_* flags when creating it.");
12025	m_Flags \|= (uint8_t)FLAG_PERSISTENT_MAP;
12026	}
12027	m_DedicatedAllocation.m_hParentPool = hParentPool;
12028	m_DedicatedAllocation.m_hMemory = hMemory;
12029	m_DedicatedAllocation.m_pMappedData = pMappedData;
12030	m_DedicatedAllocation.m_Prev = VMA_NULL;
12031	m_DedicatedAllocation.m_Next = VMA_NULL;
12032	}
12033
12034	void VmaAllocation_T::SetName(VmaAllocator hAllocator, const char* pName)
12035	{
12036	VMA_ASSERT(pName == VMA_NULL \|\| pName != m_pName);
12037
12038	FreeName(hAllocator);
12039
12040	if (pName != VMA_NULL)
12041	m_pName = VmaCreateStringCopy(hAllocator->GetAllocationCallbacks(), pName);
12042	}
12043
12044	uint8_t VmaAllocation_T::SwapBlockAllocation(VmaAllocator hAllocator, VmaAllocation allocation)
12045	{
12046	VMA_ASSERT(allocation != VMA_NULL);
12047	VMA_ASSERT(m_Type == ALLOCATION_TYPE_BLOCK);
12048	VMA_ASSERT(allocation->m_Type == ALLOCATION_TYPE_BLOCK);
12049
12050	if (m_MapCount != `0`)
12051	m_BlockAllocation.m_Block->Unmap(hAllocator, m_MapCount);
12052
12053	m_BlockAllocation.m_Block->m_pMetadata->SetAllocationUserData(m_BlockAllocation.m_AllocHandle, allocation);
12054	VMA_SWAP(m_BlockAllocation, allocation->m_BlockAllocation);
12055	m_BlockAllocation.m_Block->m_pMetadata->SetAllocationUserData(m_BlockAllocation.m_AllocHandle, this);
12056
12057	#if VMA_STATS_STRING_ENABLED
12058	VMA_SWAP(m_BufferImageUsage, allocation->m_BufferImageUsage);
12059	#endif
12060	return m_MapCount;
12061	}
12062
12063	VmaAllocHandle VmaAllocation_T::GetAllocHandle() const
12064	{
12065	switch (m_Type)
12066	{
12067	case ALLOCATION_TYPE_BLOCK:
12068	return m_BlockAllocation.m_AllocHandle;
12069	case ALLOCATION_TYPE_DEDICATED:
12070	return VK_NULL_HANDLE;
12071	default:
12072	VMA_ASSERT(`0`);
12073	return VK_NULL_HANDLE;
12074	}
12075	}
12076
12077	VkDeviceSize VmaAllocation_T::GetOffset() const
12078	{
12079	switch (m_Type)
12080	{
12081	case ALLOCATION_TYPE_BLOCK:
12082	return m_BlockAllocation.m_Block->m_pMetadata->GetAllocationOffset(m_BlockAllocation.m_AllocHandle);
12083	case ALLOCATION_TYPE_DEDICATED:
12084	return `0`;
12085	default:
12086	VMA_ASSERT(`0`);
12087	return `0`;
12088	}
12089	}
12090
12091	VmaPool VmaAllocation_T::GetParentPool() const
12092	{
12093	switch (m_Type)
12094	{
12095	case ALLOCATION_TYPE_BLOCK:
12096	return m_BlockAllocation.m_Block->GetParentPool();
12097	case ALLOCATION_TYPE_DEDICATED:
12098	return m_DedicatedAllocation.m_hParentPool;
12099	default:
12100	VMA_ASSERT(`0`);
12101	return VK_NULL_HANDLE;
12102	}
12103	}
12104
12105	VkDeviceMemory VmaAllocation_T::GetMemory() const
12106	{
12107	switch (m_Type)
12108	{
12109	case ALLOCATION_TYPE_BLOCK:
12110	return m_BlockAllocation.m_Block->GetDeviceMemory();
12111	case ALLOCATION_TYPE_DEDICATED:
12112	return m_DedicatedAllocation.m_hMemory;
12113	default:
12114	VMA_ASSERT(`0`);
12115	return VK_NULL_HANDLE;
12116	}
12117	}
12118
12119	void* VmaAllocation_T::GetMappedData() const
12120	{
12121	switch (m_Type)
12122	{
12123	case ALLOCATION_TYPE_BLOCK:
12124	if (m_MapCount != `0` \|\| IsPersistentMap())
12125	{
12126	void* pBlockData = m_BlockAllocation.m_Block->GetMappedData();
12127	VMA_ASSERT(pBlockData != VMA_NULL);
12128	return (char*)pBlockData + GetOffset();
12129	}
12130	else
12131	{
12132	return VMA_NULL;
12133	}
12134	break;
12135	case ALLOCATION_TYPE_DEDICATED:
12136	VMA_ASSERT((m_DedicatedAllocation.m_pMappedData != VMA_NULL) == (m_MapCount != `0` \|\| IsPersistentMap()));
12137	return m_DedicatedAllocation.m_pMappedData;
12138	default:
12139	VMA_ASSERT(`0`);
12140	return VMA_NULL;
12141	}
12142	}
12143
12144	void VmaAllocation_T::BlockAllocMap()
12145	{
12146	VMA_ASSERT(GetType() == ALLOCATION_TYPE_BLOCK);
12147	VMA_ASSERT(IsMappingAllowed() && "Mapping is not allowed on this allocation! Please use one of the new VMA_ALLOCATION_CREATE_HOST_ACCESS_* flags when creating it.");
12148
12149	if (m_MapCount < `0xFF`)
12150	{
12151	++m_MapCount;
12152	}
12153	else
12154	{
12155	VMA_ASSERT(`0` && "Allocation mapped too many times simultaneously.");
12156	}
12157	}
12158
12159	void VmaAllocation_T::BlockAllocUnmap()
12160	{
12161	VMA_ASSERT(GetType() == ALLOCATION_TYPE_BLOCK);
12162
12163	if (m_MapCount > `0`)
12164	{
12165	--m_MapCount;
12166	}
12167	else
12168	{
12169	VMA_ASSERT(`0` && "Unmapping allocation not previously mapped.");
12170	}
12171	}
12172
12173	VkResult VmaAllocation_T::DedicatedAllocMap(VmaAllocator hAllocator, void** ppData)
12174	{
12175	VMA_ASSERT(GetType() == ALLOCATION_TYPE_DEDICATED);
12176	VMA_ASSERT(IsMappingAllowed() && "Mapping is not allowed on this allocation! Please use one of the new VMA_ALLOCATION_CREATE_HOST_ACCESS_* flags when creating it.");
12177
12178	if (m_MapCount != `0` \|\| IsPersistentMap())
12179	{
12180	if (m_MapCount < `0xFF`)
12181	{
12182	VMA_ASSERT(m_DedicatedAllocation.m_pMappedData != VMA_NULL);
12183	*ppData = m_DedicatedAllocation.m_pMappedData;
12184	++m_MapCount;
12185	return VK_SUCCESS;
12186	}
12187	else
12188	{
12189	VMA_ASSERT(`0` && "Dedicated allocation mapped too many times simultaneously.");
12190	return VK_ERROR_MEMORY_MAP_FAILED;
12191	}
12192	}
12193	else
12194	{
12195	VkResult result = (*hAllocator->GetVulkanFunctions().vkMapMemory)(
12196	hAllocator->m_hDevice,
12197	m_DedicatedAllocation.m_hMemory,
12198	`0`, // offset
12199	VK_WHOLE_SIZE,
12200	`0`, // flags
12201	ppData);
12202	if (result == VK_SUCCESS)
12203	{
12204	m_DedicatedAllocation.m_pMappedData = *ppData;
12205	m_MapCount = `1`;
12206	}
12207	return result;
12208	}
12209	}
12210
12211	void VmaAllocation_T::DedicatedAllocUnmap(VmaAllocator hAllocator)
12212	{
12213	VMA_ASSERT(GetType() == ALLOCATION_TYPE_DEDICATED);
12214
12215	if (m_MapCount > `0`)
12216	{
12217	--m_MapCount;
12218	if (m_MapCount == `0` && !IsPersistentMap())
12219	{
12220	m_DedicatedAllocation.m_pMappedData = VMA_NULL;
12221	(*hAllocator->GetVulkanFunctions().vkUnmapMemory)(
12222	hAllocator->m_hDevice,
12223	m_DedicatedAllocation.m_hMemory);
12224	}
12225	}
12226	else
12227	{
12228	VMA_ASSERT(`0` && "Unmapping dedicated allocation not previously mapped.");
12229	}
12230	}
12231
12232	#if VMA_STATS_STRING_ENABLED
12233	void VmaAllocation_T::InitBufferImageUsage(uint32_t bufferImageUsage)
12234	{
12235	VMA_ASSERT(m_BufferImageUsage == `0`);
12236	m_BufferImageUsage = bufferImageUsage;
12237	}
12238
12239	void VmaAllocation_T::PrintParameters(class VmaJsonWriter& json) const
12240	{
12241	json.WriteString("Type");
12242	json.WriteString(VMA_SUBALLOCATION_TYPE_NAMES[m_SuballocationType]);
12243
12244	json.WriteString("Size");
12245	json.WriteNumber(m_Size);
12246	json.WriteString("Usage");
12247	json.WriteNumber(m_BufferImageUsage);
12248
12249	if (m_pUserData != VMA_NULL)
12250	{
12251	json.WriteString("CustomData");
12252	json.BeginString();
12253	json.ContinueString_Pointer(m_pUserData);
12254	json.EndString();
12255	}
12256	if (m_pName != VMA_NULL)
12257	{
12258	json.WriteString("Name");
12259	json.WriteString(m_pName);
12260	}
12261	}
12262	#endif // VMA_STATS_STRING_ENABLED
12263
12264	void VmaAllocation_T::FreeName(VmaAllocator hAllocator)
12265	{
12266	if(m_pName)
12267	{
12268	VmaFreeString(hAllocator->GetAllocationCallbacks(), m_pName);
12269	m_pName = VMA_NULL;
12270	}
12271	}
12272	#endif // _VMA_ALLOCATION_T_FUNCTIONS
12273
12274	#ifndef _VMA_BLOCK_VECTOR_FUNCTIONS
12275	VmaBlockVector::VmaBlockVector(
12276	VmaAllocator hAllocator,
12277	VmaPool hParentPool,
12278	uint32_t memoryTypeIndex,
12279	VkDeviceSize preferredBlockSize,
12280	size_t minBlockCount,
12281	size_t maxBlockCount,
12282	VkDeviceSize bufferImageGranularity,
12283	bool explicitBlockSize,
12284	uint32_t algorithm,
12285	float priority,
12286	VkDeviceSize minAllocationAlignment,
12287	void* pMemoryAllocateNext)
12288	: m_hAllocator(hAllocator),
12289	m_hParentPool(hParentPool),
12290	m_MemoryTypeIndex(memoryTypeIndex),
12291	m_PreferredBlockSize(preferredBlockSize),
12292	m_MinBlockCount(minBlockCount),
12293	m_MaxBlockCount(maxBlockCount),
12294	m_BufferImageGranularity(bufferImageGranularity),
12295	m_ExplicitBlockSize(explicitBlockSize),
12296	m_Algorithm(algorithm),
12297	m_Priority(priority),
12298	m_MinAllocationAlignment(minAllocationAlignment),
12299	m_pMemoryAllocateNext(pMemoryAllocateNext),
12300	m_Blocks(VmaStlAllocator<VmaDeviceMemoryBlock*>(hAllocator->GetAllocationCallbacks())),
12301	m_NextBlockId(`0`) {}
12302
12303	VmaBlockVector::~VmaBlockVector()
12304	{
12305	for (size_t i = m_Blocks.size(); i--; )
12306	{
12307	m_Blocks[i]->Destroy(m_hAllocator);
12308	vma_delete(m_hAllocator, m_Blocks[i]);
12309	}
12310	}
12311
12312	VkResult VmaBlockVector::CreateMinBlocks()
12313	{
12314	for (size_t i = `0`; i < m_MinBlockCount; ++i)
12315	{
12316	VkResult res = CreateBlock(m_PreferredBlockSize, VMA_NULL);
12317	if (res != VK_SUCCESS)
12318	{
12319	return res;
12320	}
12321	}
12322	return VK_SUCCESS;
12323	}
12324
12325	void VmaBlockVector::AddStatistics(VmaStatistics& inoutStats)
12326	{
12327	VmaMutexLockRead lock(m_Mutex, m_hAllocator->m_UseMutex);
12328
12329	const size_t blockCount = m_Blocks.size();
12330	for (uint32_t blockIndex = `0`; blockIndex < blockCount; ++blockIndex)
12331	{
12332	const VmaDeviceMemoryBlock* const pBlock = m_Blocks[blockIndex];
12333	VMA_ASSERT(pBlock);
12334	VMA_HEAVY_ASSERT(pBlock->Validate());
12335	pBlock->m_pMetadata->AddStatistics(inoutStats);
12336	}
12337	}
12338
12339	void VmaBlockVector::AddDetailedStatistics(VmaDetailedStatistics& inoutStats)
12340	{
12341	VmaMutexLockRead lock(m_Mutex, m_hAllocator->m_UseMutex);
12342
12343	const size_t blockCount = m_Blocks.size();
12344	for (uint32_t blockIndex = `0`; blockIndex < blockCount; ++blockIndex)
12345	{
12346	const VmaDeviceMemoryBlock* const pBlock = m_Blocks[blockIndex];
12347	VMA_ASSERT(pBlock);
12348	VMA_HEAVY_ASSERT(pBlock->Validate());
12349	pBlock->m_pMetadata->AddDetailedStatistics(inoutStats);
12350	}
12351	}
12352
12353	bool VmaBlockVector::IsEmpty()
12354	{
12355	VmaMutexLockRead lock(m_Mutex, m_hAllocator->m_UseMutex);
12356	return m_Blocks.empty();
12357	}
12358
12359	bool VmaBlockVector::IsCorruptionDetectionEnabled() const
12360	{
12361	const uint32_t requiredMemFlags = VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT \| VK_MEMORY_PROPERTY_HOST_COHERENT_BIT;
12362	return (VMA_DEBUG_DETECT_CORRUPTION != `0`) &&
12363	(VMA_DEBUG_MARGIN > `0`) &&
12364	(m_Algorithm == `0` \|\| m_Algorithm == VMA_POOL_CREATE_LINEAR_ALGORITHM_BIT) &&
12365	(m_hAllocator->m_MemProps.memoryTypes[m_MemoryTypeIndex].propertyFlags & requiredMemFlags) == requiredMemFlags;
12366	}
12367
12368	VkResult VmaBlockVector::Allocate(
12369	VkDeviceSize size,
12370	VkDeviceSize alignment,
12371	const VmaAllocationCreateInfo& createInfo,
12372	VmaSuballocationType suballocType,
12373	size_t allocationCount,
12374	VmaAllocation* pAllocations)
12375	{
12376	size_t allocIndex;
12377	VkResult res = VK_SUCCESS;
12378
12379	alignment = VMA_MAX(alignment, m_MinAllocationAlignment);
12380
12381	if (IsCorruptionDetectionEnabled())
12382	{
12383	size = VmaAlignUp<VkDeviceSize>(size, sizeof(VMA_CORRUPTION_DETECTION_MAGIC_VALUE));
12384	alignment = VmaAlignUp<VkDeviceSize>(alignment, sizeof(VMA_CORRUPTION_DETECTION_MAGIC_VALUE));
12385	}
12386
12387	{
12388	VmaMutexLockWrite lock(m_Mutex, m_hAllocator->m_UseMutex);
12389	for (allocIndex = `0`; allocIndex < allocationCount; ++allocIndex)
12390	{
12391	res = AllocatePage(
12392	size,
12393	alignment,
12394	createInfo,
12395	suballocType,
12396	pAllocations + allocIndex);
12397	if (res != VK_SUCCESS)
12398	{
12399	break;
12400	}
12401	}
12402	}
12403
12404	if (res != VK_SUCCESS)
12405	{
12406	// Free all already created allocations.
12407	while (allocIndex--)
12408	Free(pAllocations[allocIndex]);
12409	memset(pAllocations, `0`, sizeof(VmaAllocation) * allocationCount);
12410	}
12411
12412	return res;
12413	}
12414
12415	VkResult VmaBlockVector::AllocatePage(
12416	VkDeviceSize size,
12417	VkDeviceSize alignment,
12418	const VmaAllocationCreateInfo& createInfo,
12419	VmaSuballocationType suballocType,
12420	VmaAllocation* pAllocation)
12421	{
12422	const bool isUpperAddress = (createInfo.flags & VMA_ALLOCATION_CREATE_UPPER_ADDRESS_BIT) != `0`;
12423
12424	VkDeviceSize freeMemory;
12425	{
12426	const uint32_t heapIndex = m_hAllocator->MemoryTypeIndexToHeapIndex(m_MemoryTypeIndex);
12427	VmaBudget heapBudget = {};
12428	m_hAllocator->GetHeapBudgets(&heapBudget, heapIndex, `1`);
12429	freeMemory = (heapBudget.usage < heapBudget.budget) ? (heapBudget.budget - heapBudget.usage) : `0`;
12430	}
12431
12432	const bool canFallbackToDedicated = !HasExplicitBlockSize() &&
12433	(createInfo.flags & VMA_ALLOCATION_CREATE_NEVER_ALLOCATE_BIT) == `0`;
12434	const bool canCreateNewBlock =
12435	((createInfo.flags & VMA_ALLOCATION_CREATE_NEVER_ALLOCATE_BIT) == `0`) &&
12436	(m_Blocks.size() < m_MaxBlockCount) &&
12437	(freeMemory >= size \|\| !canFallbackToDedicated);
12438	uint32_t strategy = createInfo.flags & VMA_ALLOCATION_CREATE_STRATEGY_MASK;
12439
12440	// Upper address can only be used with linear allocator and within single memory block.
12441	if (isUpperAddress &&
12442	(m_Algorithm != VMA_POOL_CREATE_LINEAR_ALGORITHM_BIT \|\| m_MaxBlockCount > `1`))
12443	{
12444	return VK_ERROR_FEATURE_NOT_PRESENT;
12445	}
12446
12447	// Early reject: requested allocation size is larger that maximum block size for this block vector.
12448	if (size + VMA_DEBUG_MARGIN > m_PreferredBlockSize)
12449	{
12450	return VK_ERROR_OUT_OF_DEVICE_MEMORY;
12451	}
12452
12453	// 1. Search existing allocations. Try to allocate.
12454	if (m_Algorithm == VMA_POOL_CREATE_LINEAR_ALGORITHM_BIT)
12455	{
12456	// Use only last block.
12457	if (!m_Blocks.empty())
12458	{
12459	VmaDeviceMemoryBlock* const pCurrBlock = m_Blocks.back();
12460	VMA_ASSERT(pCurrBlock);
12461	VkResult res = AllocateFromBlock(
12462	pCurrBlock, size, alignment, createInfo.flags, createInfo.pUserData, suballocType, strategy, pAllocation);
12463	if (res == VK_SUCCESS)
12464	{
12465	VMA_DEBUG_LOG(" Returned from last block #%u", pCurrBlock->GetId());
12466	IncrementallySortBlocks();
12467	return VK_SUCCESS;
12468	}
12469	}
12470	}
12471	else
12472	{
12473	if (strategy != VMA_ALLOCATION_CREATE_STRATEGY_MIN_TIME_BIT) // MIN_MEMORY or default
12474	{
12475	const bool isHostVisible =
12476	(m_hAllocator->m_MemProps.memoryTypes[m_MemoryTypeIndex].propertyFlags & VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT) != `0`;
12477	if(isHostVisible)
12478	{
12479	const bool isMappingAllowed = (createInfo.flags &
12480	(VMA_ALLOCATION_CREATE_HOST_ACCESS_SEQUENTIAL_WRITE_BIT \| VMA_ALLOCATION_CREATE_HOST_ACCESS_RANDOM_BIT)) != `0`;
12481	/*
12482	For non-mappable allocations, check blocks that are not mapped first.
12483	For mappable allocations, check blocks that are already mapped first.
12484	This way, having many blocks, we will separate mappable and non-mappable allocations,
12485	hopefully limiting the number of blocks that are mapped, which will help tools like RenderDoc.
12486	*/
12487	for(size_t mappingI = `0`; mappingI < `2`; ++mappingI)
12488	{
12489	// Forward order in m_Blocks - prefer blocks with smallest amount of free space.
12490	for (size_t blockIndex = `0`; blockIndex < m_Blocks.size(); ++blockIndex)
12491	{
12492	VmaDeviceMemoryBlock* const pCurrBlock = m_Blocks[blockIndex];
12493	VMA_ASSERT(pCurrBlock);
12494	const bool isBlockMapped = pCurrBlock->GetMappedData() != VMA_NULL;
12495	if((mappingI == `0`) == (isMappingAllowed == isBlockMapped))
12496	{
12497	VkResult res = AllocateFromBlock(
12498	pCurrBlock, size, alignment, createInfo.flags, createInfo.pUserData, suballocType, strategy, pAllocation);
12499	if (res == VK_SUCCESS)
12500	{
12501	VMA_DEBUG_LOG(" Returned from existing block #%u", pCurrBlock->GetId());
12502	IncrementallySortBlocks();
12503	return VK_SUCCESS;
12504	}
12505	}
12506	}
12507	}
12508	}
12509	else
12510	{
12511	// Forward order in m_Blocks - prefer blocks with smallest amount of free space.
12512	for (size_t blockIndex = `0`; blockIndex < m_Blocks.size(); ++blockIndex)
12513	{
12514	VmaDeviceMemoryBlock* const pCurrBlock = m_Blocks[blockIndex];
12515	VMA_ASSERT(pCurrBlock);
12516	VkResult res = AllocateFromBlock(
12517	pCurrBlock, size, alignment, createInfo.flags, createInfo.pUserData, suballocType, strategy, pAllocation);
12518	if (res == VK_SUCCESS)
12519	{
12520	VMA_DEBUG_LOG(" Returned from existing block #%u", pCurrBlock->GetId());
12521	IncrementallySortBlocks();
12522	return VK_SUCCESS;
12523	}
12524	}
12525	}
12526	}
12527	else // VMA_ALLOCATION_CREATE_STRATEGY_MIN_TIME_BIT
12528	{
12529	// Backward order in m_Blocks - prefer blocks with largest amount of free space.
12530	for (size_t blockIndex = m_Blocks.size(); blockIndex--; )
12531	{
12532	VmaDeviceMemoryBlock* const pCurrBlock = m_Blocks[blockIndex];
12533	VMA_ASSERT(pCurrBlock);
12534	VkResult res = AllocateFromBlock(pCurrBlock, size, alignment, createInfo.flags, createInfo.pUserData, suballocType, strategy, pAllocation);
12535	if (res == VK_SUCCESS)
12536	{
12537	VMA_DEBUG_LOG(" Returned from existing block #%u", pCurrBlock->GetId());
12538	IncrementallySortBlocks();
12539	return VK_SUCCESS;
12540	}
12541	}
12542	}
12543	}
12544
12545	// 2. Try to create new block.
12546	if (canCreateNewBlock)
12547	{
12548	// Calculate optimal size for new block.
12549	VkDeviceSize newBlockSize = m_PreferredBlockSize;
12550	uint32_t newBlockSizeShift = `0`;
12551	const uint32_t NEW_BLOCK_SIZE_SHIFT_MAX = `3`;
12552
12553	if (!m_ExplicitBlockSize)
12554	{
12555	// Allocate 1/8, 1/4, 1/2 as first blocks.
12556	const VkDeviceSize maxExistingBlockSize = CalcMaxBlockSize();
12557	for (uint32_t i = `0`; i < NEW_BLOCK_SIZE_SHIFT_MAX; ++i)
12558	{
12559	const VkDeviceSize smallerNewBlockSize = newBlockSize / `2`;
12560	if (smallerNewBlockSize > maxExistingBlockSize && smallerNewBlockSize >= size * `2`)
12561	{
12562	newBlockSize = smallerNewBlockSize;
12563	++newBlockSizeShift;
12564	}
12565	else
12566	{
12567	break;
12568	}
12569	}
12570	}
12571
12572	size_t newBlockIndex = `0`;
12573	VkResult res = (newBlockSize <= freeMemory \|\| !canFallbackToDedicated) ?
12574	CreateBlock(newBlockSize, &newBlockIndex) : VK_ERROR_OUT_OF_DEVICE_MEMORY;
12575	// Allocation of this size failed? Try 1/2, 1/4, 1/8 of m_PreferredBlockSize.
12576	if (!m_ExplicitBlockSize)
12577	{
12578	while (res < `0` && newBlockSizeShift < NEW_BLOCK_SIZE_SHIFT_MAX)
12579	{
12580	const VkDeviceSize smallerNewBlockSize = newBlockSize / `2`;
12581	if (smallerNewBlockSize >= size)
12582	{
12583	newBlockSize = smallerNewBlockSize;
12584	++newBlockSizeShift;
12585	res = (newBlockSize <= freeMemory \|\| !canFallbackToDedicated) ?
12586	CreateBlock(newBlockSize, &newBlockIndex) : VK_ERROR_OUT_OF_DEVICE_MEMORY;
12587	}
12588	else
12589	{
12590	break;
12591	}
12592	}
12593	}
12594
12595	if (res == VK_SUCCESS)
12596	{
12597	VmaDeviceMemoryBlock* const pBlock = m_Blocks[newBlockIndex];
12598	VMA_ASSERT(pBlock->m_pMetadata->GetSize() >= size);
12599
12600	res = AllocateFromBlock(
12601	pBlock, size, alignment, createInfo.flags, createInfo.pUserData, suballocType, strategy, pAllocation);
12602	if (res == VK_SUCCESS)
12603	{
12604	VMA_DEBUG_LOG(" Created new block #%u Size=%llu", pBlock->GetId(), newBlockSize);
12605	IncrementallySortBlocks();
12606	return VK_SUCCESS;
12607	}
12608	else
12609	{
12610	// Allocation from new block failed, possibly due to VMA_DEBUG_MARGIN or alignment.
12611	return VK_ERROR_OUT_OF_DEVICE_MEMORY;
12612	}
12613	}
12614	}
12615
12616	return VK_ERROR_OUT_OF_DEVICE_MEMORY;
12617	}
12618
12619	void VmaBlockVector::Free(const VmaAllocation hAllocation)
12620	{
12621	VmaDeviceMemoryBlock* pBlockToDelete = VMA_NULL;
12622
12623	bool budgetExceeded = false;
12624	{
12625	const uint32_t heapIndex = m_hAllocator->MemoryTypeIndexToHeapIndex(m_MemoryTypeIndex);
12626	VmaBudget heapBudget = {};
12627	m_hAllocator->GetHeapBudgets(&heapBudget, heapIndex, `1`);
12628	budgetExceeded = heapBudget.usage >= heapBudget.budget;
12629	}
12630
12631	// Scope for lock.
12632	{
12633	VmaMutexLockWrite lock(m_Mutex, m_hAllocator->m_UseMutex);
12634
12635	VmaDeviceMemoryBlock* pBlock = hAllocation->GetBlock();
12636
12637	if (IsCorruptionDetectionEnabled())
12638	{
12639	VkResult res = pBlock->ValidateMagicValueAfterAllocation(m_hAllocator, hAllocation->GetOffset(), hAllocation->GetSize());
12640	VMA_ASSERT(res == VK_SUCCESS && "Couldn't map block memory to validate magic value.");
12641	}
12642
12643	if (hAllocation->IsPersistentMap())
12644	{
12645	pBlock->Unmap(m_hAllocator, `1`);
12646	}
12647
12648	const bool hadEmptyBlockBeforeFree = HasEmptyBlock();
12649	pBlock->m_pMetadata->Free(hAllocation->GetAllocHandle());
12650	pBlock->PostFree(m_hAllocator);
12651	VMA_HEAVY_ASSERT(pBlock->Validate());
12652
12653	VMA_DEBUG_LOG(" Freed from MemoryTypeIndex=%u", m_MemoryTypeIndex);
12654
12655	const bool canDeleteBlock = m_Blocks.size() > m_MinBlockCount;
12656	// pBlock became empty after this deallocation.
12657	if (pBlock->m_pMetadata->IsEmpty())
12658	{
12659	// Already had empty block. We don't want to have two, so delete this one.
12660	if ((hadEmptyBlockBeforeFree \|\| budgetExceeded) && canDeleteBlock)
12661	{
12662	pBlockToDelete = pBlock;
12663	Remove(pBlock);
12664	}
12665	// else: We now have one empty block - leave it. A hysteresis to avoid allocating whole block back and forth.
12666	}
12667	// pBlock didn't become empty, but we have another empty block - find and free that one.
12668	// (This is optional, heuristics.)
12669	else if (hadEmptyBlockBeforeFree && canDeleteBlock)
12670	{
12671	VmaDeviceMemoryBlock* pLastBlock = m_Blocks.back();
12672	if (pLastBlock->m_pMetadata->IsEmpty())
12673	{
12674	pBlockToDelete = pLastBlock;
12675	m_Blocks.pop_back();
12676	}
12677	}
12678
12679	IncrementallySortBlocks();
12680	}
12681
12682	// Destruction of a free block. Deferred until this point, outside of mutex
12683	// lock, for performance reason.
12684	if (pBlockToDelete != VMA_NULL)
12685	{
12686	VMA_DEBUG_LOG(" Deleted empty block #%u", pBlockToDelete->GetId());
12687	pBlockToDelete->Destroy(m_hAllocator);
12688	vma_delete(m_hAllocator, pBlockToDelete);
12689	}
12690
12691	m_hAllocator->m_Budget.RemoveAllocation(m_hAllocator->MemoryTypeIndexToHeapIndex(m_MemoryTypeIndex), hAllocation->GetSize());
12692	m_hAllocator->m_AllocationObjectAllocator.Free(hAllocation);
12693	}
12694
12695	VkDeviceSize VmaBlockVector::CalcMaxBlockSize() const
12696	{
12697	VkDeviceSize result = `0`;
12698	for (size_t i = m_Blocks.size(); i--; )
12699	{
12700	result = VMA_MAX(result, m_Blocks[i]->m_pMetadata->GetSize());
12701	if (result >= m_PreferredBlockSize)
12702	{
12703	break;
12704	}
12705	}
12706	return result;
12707	}
12708
12709	void VmaBlockVector::Remove(VmaDeviceMemoryBlock* pBlock)
12710	{
12711	for (uint32_t blockIndex = `0`; blockIndex < m_Blocks.size(); ++blockIndex)
12712	{
12713	if (m_Blocks[blockIndex] == pBlock)
12714	{
12715	VmaVectorRemove(m_Blocks, blockIndex);
12716	return;
12717	}
12718	}
12719	VMA_ASSERT(`0`);
12720	}
12721
12722	void VmaBlockVector::IncrementallySortBlocks()
12723	{
12724	if (!m_IncrementalSort)
12725	return;
12726	if (m_Algorithm != VMA_POOL_CREATE_LINEAR_ALGORITHM_BIT)
12727	{
12728	// Bubble sort only until first swap.
12729	for (size_t i = `1`; i < m_Blocks.size(); ++i)
12730	{
12731	if (m_Blocks[i - `1`]->m_pMetadata->GetSumFreeSize() > m_Blocks[i]->m_pMetadata->GetSumFreeSize())
12732	{
12733	VMA_SWAP(m_Blocks[i - `1`], m_Blocks[i]);
12734	return;
12735	}
12736	}
12737	}
12738	}
12739
12740	void VmaBlockVector::SortByFreeSize()
12741	{
12742	VMA_SORT(m_Blocks.begin(), m_Blocks.end(),
12743	[](VmaDeviceMemoryBlock* b1, VmaDeviceMemoryBlock* b2) -> bool
12744	{
12745	return b1->m_pMetadata->GetSumFreeSize() < b2->m_pMetadata->GetSumFreeSize();
12746	});
12747	}
12748
12749	VkResult VmaBlockVector::AllocateFromBlock(
12750	VmaDeviceMemoryBlock* pBlock,
12751	VkDeviceSize size,
12752	VkDeviceSize alignment,
12753	VmaAllocationCreateFlags allocFlags,
12754	void* pUserData,
12755	VmaSuballocationType suballocType,
12756	uint32_t strategy,
12757	VmaAllocation* pAllocation)
12758	{
12759	const bool isUpperAddress = (allocFlags & VMA_ALLOCATION_CREATE_UPPER_ADDRESS_BIT) != `0`;
12760
12761	VmaAllocationRequest currRequest = {};
12762	if (pBlock->m_pMetadata->CreateAllocationRequest(
12763	size,
12764	alignment,
12765	isUpperAddress,
12766	suballocType,
12767	strategy,
12768	&currRequest))
12769	{
12770	return CommitAllocationRequest(currRequest, pBlock, alignment, allocFlags, pUserData, suballocType, pAllocation);
12771	}
12772	return VK_ERROR_OUT_OF_DEVICE_MEMORY;
12773	}
12774
12775	VkResult VmaBlockVector::CommitAllocationRequest(
12776	VmaAllocationRequest& allocRequest,
12777	VmaDeviceMemoryBlock* pBlock,
12778	VkDeviceSize alignment,
12779	VmaAllocationCreateFlags allocFlags,
12780	void* pUserData,
12781	VmaSuballocationType suballocType,
12782	VmaAllocation* pAllocation)
12783	{
12784	const bool mapped = (allocFlags & VMA_ALLOCATION_CREATE_MAPPED_BIT) != `0`;
12785	const bool isUserDataString = (allocFlags & VMA_ALLOCATION_CREATE_USER_DATA_COPY_STRING_BIT) != `0`;
12786	const bool isMappingAllowed = (allocFlags &
12787	(VMA_ALLOCATION_CREATE_HOST_ACCESS_SEQUENTIAL_WRITE_BIT \| VMA_ALLOCATION_CREATE_HOST_ACCESS_RANDOM_BIT)) != `0`;
12788
12789	pBlock->PostAlloc();
12790	// Allocate from pCurrBlock.
12791	if (mapped)
12792	{
12793	VkResult res = pBlock->Map(m_hAllocator, `1`, VMA_NULL);
12794	if (res != VK_SUCCESS)
12795	{
12796	return res;
12797	}
12798	}
12799
12800	*pAllocation = m_hAllocator->m_AllocationObjectAllocator.Allocate(isMappingAllowed);
12801	pBlock->m_pMetadata->Alloc(allocRequest, suballocType, *pAllocation);
12802	(*pAllocation)->InitBlockAllocation(
12803	pBlock,
12804	allocRequest.allocHandle,
12805	alignment,
12806	allocRequest.size, // Not size, as actual allocation size may be larger than requested!
12807	m_MemoryTypeIndex,
12808	suballocType,
12809	mapped);
12810	VMA_HEAVY_ASSERT(pBlock->Validate());
12811	if (isUserDataString)
12812	(pAllocation)->SetName(m_hAllocator, (const* char*)pUserData);
12813	else
12814	(*pAllocation)->SetUserData(m_hAllocator, pUserData);
12815	m_hAllocator->m_Budget.AddAllocation(m_hAllocator->MemoryTypeIndexToHeapIndex(m_MemoryTypeIndex), allocRequest.size);
12816	if (VMA_DEBUG_INITIALIZE_ALLOCATIONS)
12817	{
12818	m_hAllocator->FillAllocation(*pAllocation, VMA_ALLOCATION_FILL_PATTERN_CREATED);
12819	}
12820	if (IsCorruptionDetectionEnabled())
12821	{
12822	VkResult res = pBlock->WriteMagicValueAfterAllocation(m_hAllocator, (*pAllocation)->GetOffset(), allocRequest.size);
12823	VMA_ASSERT(res == VK_SUCCESS && "Couldn't map block memory to write magic value.");
12824	}
12825	return VK_SUCCESS;
12826	}
12827
12828	VkResult VmaBlockVector::CreateBlock(VkDeviceSize blockSize, size_t* pNewBlockIndex)
12829	{
12830	VkMemoryAllocateInfo allocInfo = { VK_STRUCTURE_TYPE_MEMORY_ALLOCATE_INFO };
12831	allocInfo.pNext = m_pMemoryAllocateNext;
12832	allocInfo.memoryTypeIndex = m_MemoryTypeIndex;
12833	allocInfo.allocationSize = blockSize;
12834
12835	#if VMA_BUFFER_DEVICE_ADDRESS
12836	// Every standalone block can potentially contain a buffer with VK_BUFFER_USAGE_SHADER_DEVICE_ADDRESS_BIT - always enable the feature.
12837	VkMemoryAllocateFlagsInfoKHR allocFlagsInfo = { VK_STRUCTURE_TYPE_MEMORY_ALLOCATE_FLAGS_INFO_KHR };
12838	if (m_hAllocator->m_UseKhrBufferDeviceAddress)
12839	{
12840	allocFlagsInfo.flags = VK_MEMORY_ALLOCATE_DEVICE_ADDRESS_BIT_KHR;
12841	VmaPnextChainPushFront(&allocInfo, &allocFlagsInfo);
12842	}
12843	#endif // VMA_BUFFER_DEVICE_ADDRESS
12844
12845	#if VMA_MEMORY_PRIORITY
12846	VkMemoryPriorityAllocateInfoEXT priorityInfo = { VK_STRUCTURE_TYPE_MEMORY_PRIORITY_ALLOCATE_INFO_EXT };
12847	if (m_hAllocator->m_UseExtMemoryPriority)
12848	{
12849	VMA_ASSERT(m_Priority >= `0.f` && m_Priority <= `1.f`);
12850	priorityInfo.priority = m_Priority;
12851	VmaPnextChainPushFront(&allocInfo, &priorityInfo);
12852	}
12853	#endif // VMA_MEMORY_PRIORITY
12854
12855	#if VMA_EXTERNAL_MEMORY
12856	// Attach VkExportMemoryAllocateInfoKHR if necessary.
12857	VkExportMemoryAllocateInfoKHR exportMemoryAllocInfo = { VK_STRUCTURE_TYPE_EXPORT_MEMORY_ALLOCATE_INFO_KHR };
12858	exportMemoryAllocInfo.handleTypes = m_hAllocator->GetExternalMemoryHandleTypeFlags(m_MemoryTypeIndex);
12859	if (exportMemoryAllocInfo.handleTypes != `0`)
12860	{
12861	VmaPnextChainPushFront(&allocInfo, &exportMemoryAllocInfo);
12862	}
12863	#endif // VMA_EXTERNAL_MEMORY
12864
12865	VkDeviceMemory mem = VK_NULL_HANDLE;
12866	VkResult res = m_hAllocator->AllocateVulkanMemory(&allocInfo, &mem);
12867	if (res < `0`)
12868	{
12869	return res;
12870	}
12871
12872	// New VkDeviceMemory successfully created.
12873
12874	// Create new Allocation for it.
12875	VmaDeviceMemoryBlock* const pBlock = vma_new(m_hAllocator, VmaDeviceMemoryBlock)(m_hAllocator);
12876	pBlock->Init(
12877	m_hAllocator,
12878	m_hParentPool,
12879	m_MemoryTypeIndex,
12880	mem,
12881	allocInfo.allocationSize,
12882	m_NextBlockId++,
12883	m_Algorithm,
12884	m_BufferImageGranularity);
12885
12886	m_Blocks.push_back(pBlock);
12887	if (pNewBlockIndex != VMA_NULL)
12888	{
12889	*pNewBlockIndex = m_Blocks.size() - `1`;
12890	}
12891
12892	return VK_SUCCESS;
12893	}
12894
12895	bool VmaBlockVector::HasEmptyBlock()
12896	{
12897	for (size_t index = `0`, count = m_Blocks.size(); index < count; ++index)
12898	{
12899	VmaDeviceMemoryBlock* const pBlock = m_Blocks[index];
12900	if (pBlock->m_pMetadata->IsEmpty())
12901	{
12902	return true;
12903	}
12904	}
12905	return false;
12906	}
12907
12908	#if VMA_STATS_STRING_ENABLED
12909	void VmaBlockVector::PrintDetailedMap(class VmaJsonWriter& json)
12910	{
12911	VmaMutexLockRead lock(m_Mutex, m_hAllocator->m_UseMutex);
12912
12913
12914	json.BeginObject();
12915	for (size_t i = `0`; i < m_Blocks.size(); ++i)
12916	{
12917	json.BeginString();
12918	json.ContinueString(m_Blocks[i]->GetId());
12919	json.EndString();
12920
12921	json.BeginObject();
12922	json.WriteString("MapRefCount");
12923	json.WriteNumber(m_Blocks[i]->GetMapRefCount());
12924
12925	m_Blocks[i]->m_pMetadata->PrintDetailedMap(json);
12926	json.EndObject();
12927	}
12928	json.EndObject();
12929	}
12930	#endif // VMA_STATS_STRING_ENABLED
12931
12932	VkResult VmaBlockVector::CheckCorruption()
12933	{
12934	if (!IsCorruptionDetectionEnabled())
12935	{
12936	return VK_ERROR_FEATURE_NOT_PRESENT;
12937	}
12938
12939	VmaMutexLockRead lock(m_Mutex, m_hAllocator->m_UseMutex);
12940	for (uint32_t blockIndex = `0`; blockIndex < m_Blocks.size(); ++blockIndex)
12941	{
12942	VmaDeviceMemoryBlock* const pBlock = m_Blocks[blockIndex];
12943	VMA_ASSERT(pBlock);
12944	VkResult res = pBlock->CheckCorruption(m_hAllocator);
12945	if (res != VK_SUCCESS)
12946	{
12947	return res;
12948	}
12949	}
12950	return VK_SUCCESS;
12951	}
12952
12953	#endif // _VMA_BLOCK_VECTOR_FUNCTIONS
12954
12955	#ifndef _VMA_DEFRAGMENTATION_CONTEXT_FUNCTIONS
12956	VmaDefragmentationContext_T::VmaDefragmentationContext_T(
12957	VmaAllocator hAllocator,
12958	const VmaDefragmentationInfo& info)
12959	: m_MaxPassBytes(info.maxBytesPerPass == `0` ? VK_WHOLE_SIZE : info.maxBytesPerPass),
12960	m_MaxPassAllocations(info.maxAllocationsPerPass == `0` ? UINT32_MAX : info.maxAllocationsPerPass),
12961	m_MoveAllocator(hAllocator->GetAllocationCallbacks()),
12962	m_Moves(m_MoveAllocator)
12963	{
12964	m_Algorithm = info.flags & VMA_DEFRAGMENTATION_FLAG_ALGORITHM_MASK;
12965
12966	if (info.pool != VMA_NULL)
12967	{
12968	m_BlockVectorCount = `1`;
12969	m_PoolBlockVector = &info.pool->m_BlockVector;
12970	m_pBlockVectors = &m_PoolBlockVector;
12971	m_PoolBlockVector->SetIncrementalSort(false);
12972	m_PoolBlockVector->SortByFreeSize();
12973	}
12974	else
12975	{
12976	m_BlockVectorCount = hAllocator->GetMemoryTypeCount();
12977	m_PoolBlockVector = VMA_NULL;
12978	m_pBlockVectors = hAllocator->m_pBlockVectors;
12979	for (uint32_t i = `0`; i < m_BlockVectorCount; ++i)
12980	{
12981	VmaBlockVector* vector = m_pBlockVectors[i];
12982	if (vector != VMA_NULL)
12983	{
12984	vector->SetIncrementalSort(false);
12985	vector->SortByFreeSize();
12986	}
12987	}
12988	}
12989
12990	switch (m_Algorithm)
12991	{
12992	case `0`: // Default algorithm
12993	m_Algorithm = VMA_DEFRAGMENTATION_FLAG_ALGORITHM_BALANCED_BIT;
12994	case VMA_DEFRAGMENTATION_FLAG_ALGORITHM_BALANCED_BIT:
12995	{
12996	m_AlgorithmState = vma_new_array(hAllocator, StateBalanced, m_BlockVectorCount);
12997	break;
12998	}
12999	case VMA_DEFRAGMENTATION_FLAG_ALGORITHM_EXTENSIVE_BIT:
13000	{
13001	if (hAllocator->GetBufferImageGranularity() > `1`)
13002	{
13003	m_AlgorithmState = vma_new_array(hAllocator, StateExtensive, m_BlockVectorCount);
13004	}
13005	break;
13006	}
13007	}
13008	}
13009
13010	VmaDefragmentationContext_T::~VmaDefragmentationContext_T()
13011	{
13012	if (m_PoolBlockVector != VMA_NULL)
13013	{
13014	m_PoolBlockVector->SetIncrementalSort(true);
13015	}
13016	else
13017	{
13018	for (uint32_t i = `0`; i < m_BlockVectorCount; ++i)
13019	{
13020	VmaBlockVector* vector = m_pBlockVectors[i];
13021	if (vector != VMA_NULL)
13022	vector->SetIncrementalSort(true);
13023	}
13024	}
13025
13026	if (m_AlgorithmState)
13027	{
13028	switch (m_Algorithm)
13029	{
13030	case VMA_DEFRAGMENTATION_FLAG_ALGORITHM_BALANCED_BIT:
13031	vma_delete_array(m_MoveAllocator.m_pCallbacks, reinterpret_cast<StateBalanced*>(m_AlgorithmState), m_BlockVectorCount);
13032	break;
13033	case VMA_DEFRAGMENTATION_FLAG_ALGORITHM_EXTENSIVE_BIT:
13034	vma_delete_array(m_MoveAllocator.m_pCallbacks, reinterpret_cast<StateExtensive*>(m_AlgorithmState), m_BlockVectorCount);
13035	break;
13036	default:
13037	VMA_ASSERT(`0`);
13038	}
13039	}
13040	}
13041
13042	VkResult VmaDefragmentationContext_T::DefragmentPassBegin(VmaDefragmentationPassMoveInfo& moveInfo)
13043	{
13044	if (m_PoolBlockVector != VMA_NULL)
13045	{
13046	VmaMutexLockWrite lock(m_PoolBlockVector->GetMutex(), m_PoolBlockVector->GetAllocator()->m_UseMutex);
13047
13048	if (m_PoolBlockVector->GetBlockCount() > `1`)
13049	ComputeDefragmentation(*m_PoolBlockVector, `0`);
13050	else if (m_PoolBlockVector->GetBlockCount() == `1`)
13051	ReallocWithinBlock(*m_PoolBlockVector, m_PoolBlockVector->GetBlock(`0`));
13052	}
13053	else
13054	{
13055	for (uint32_t i = `0`; i < m_BlockVectorCount; ++i)
13056	{
13057	if (m_pBlockVectors[i] != VMA_NULL)
13058	{
13059	VmaMutexLockWrite lock(m_pBlockVectors[i]->GetMutex(), m_pBlockVectors[i]->GetAllocator()->m_UseMutex);
13060
13061	if (m_pBlockVectors[i]->GetBlockCount() > `1`)
13062	{
13063	if (ComputeDefragmentation(*m_pBlockVectors[i], i))
13064	break;
13065	}
13066	else if (m_pBlockVectors[i]->GetBlockCount() == `1`)
13067	{
13068	if (ReallocWithinBlock(*m_pBlockVectors[i], m_pBlockVectors[i]->GetBlock(`0`)))
13069	break;
13070	}
13071	}
13072	}
13073	}
13074
13075	moveInfo.moveCount = static_cast<uint32_t>(m_Moves.size());
13076	if (moveInfo.moveCount > `0`)
13077	{
13078	moveInfo.pMoves = m_Moves.data();
13079	return VK_INCOMPLETE;
13080	}
13081
13082	moveInfo.pMoves = VMA_NULL;
13083	return VK_SUCCESS;
13084	}
13085
13086	VkResult VmaDefragmentationContext_T::DefragmentPassEnd(VmaDefragmentationPassMoveInfo& moveInfo)
13087	{
13088	VMA_ASSERT(moveInfo.moveCount > `0` ? moveInfo.pMoves != VMA_NULL : true);
13089
13090	VkResult result = VK_SUCCESS;
13091	VmaStlAllocator<FragmentedBlock> blockAllocator(m_MoveAllocator.m_pCallbacks);
13092	VmaVector<FragmentedBlock, VmaStlAllocator<FragmentedBlock>> immovableBlocks(blockAllocator);
13093	VmaVector<FragmentedBlock, VmaStlAllocator<FragmentedBlock>> mappedBlocks(blockAllocator);
13094
13095	VmaAllocator allocator = VMA_NULL;
13096	for (uint32_t i = `0`; i < moveInfo.moveCount; ++i)
13097	{
13098	VmaDefragmentationMove& move = moveInfo.pMoves[i];
13099	size_t prevCount = `0`, currentCount = `0`;
13100	VkDeviceSize freedBlockSize = `0`;
13101
13102	uint32_t vectorIndex;
13103	VmaBlockVector* vector;
13104	if (m_PoolBlockVector != VMA_NULL)
13105	{
13106	vectorIndex = `0`;
13107	vector = m_PoolBlockVector;
13108	}
13109	else
13110	{
13111	vectorIndex = move.srcAllocation->GetMemoryTypeIndex();
13112	vector = m_pBlockVectors[vectorIndex];
13113	VMA_ASSERT(vector != VMA_NULL);
13114	}
13115
13116	switch (move.operation)
13117	{
13118	case VMA_DEFRAGMENTATION_MOVE_OPERATION_COPY:
13119	{
13120	uint8_t mapCount = move.srcAllocation->SwapBlockAllocation(vector->m_hAllocator, move.dstTmpAllocation);
13121	if (mapCount > `0`)
13122	{
13123	allocator = vector->m_hAllocator;
13124	VmaDeviceMemoryBlock* newMapBlock = move.srcAllocation->GetBlock();
13125	bool notPresent = true;
13126	for (FragmentedBlock& block : mappedBlocks)
13127	{
13128	if (block.block == newMapBlock)
13129	{
13130	notPresent = false;
13131	block.data += mapCount;
13132	break;
13133	}
13134	}
13135	if (notPresent)
13136	mappedBlocks.push_back({ mapCount, newMapBlock });
13137	}
13138
13139	// Scope for locks, Free have it's own lock
13140	{
13141	VmaMutexLockRead lock(vector->GetMutex(), vector->GetAllocator()->m_UseMutex);
13142	prevCount = vector->GetBlockCount();
13143	freedBlockSize = move.dstTmpAllocation->GetBlock()->m_pMetadata->GetSize();
13144	}
13145	vector->Free(move.dstTmpAllocation);
13146	{
13147	VmaMutexLockRead lock(vector->GetMutex(), vector->GetAllocator()->m_UseMutex);
13148	currentCount = vector->GetBlockCount();
13149	}
13150
13151	result = VK_INCOMPLETE;
13152	break;
13153	}
13154	case VMA_DEFRAGMENTATION_MOVE_OPERATION_IGNORE:
13155	{
13156	m_PassStats.bytesMoved -= move.srcAllocation->GetSize();
13157	--m_PassStats.allocationsMoved;
13158	vector->Free(move.dstTmpAllocation);
13159
13160	VmaDeviceMemoryBlock* newBlock = move.srcAllocation->GetBlock();
13161	bool notPresent = true;
13162	for (const FragmentedBlock& block : immovableBlocks)
13163	{
13164	if (block.block == newBlock)
13165	{
13166	notPresent = false;
13167	break;
13168	}
13169	}
13170	if (notPresent)
13171	immovableBlocks.push_back({ vectorIndex, newBlock });
13172	break;
13173	}
13174	case VMA_DEFRAGMENTATION_MOVE_OPERATION_DESTROY:
13175	{
13176	m_PassStats.bytesMoved -= move.srcAllocation->GetSize();
13177	--m_PassStats.allocationsMoved;
13178	// Scope for locks, Free have it's own lock
13179	{
13180	VmaMutexLockRead lock(vector->GetMutex(), vector->GetAllocator()->m_UseMutex);
13181	prevCount = vector->GetBlockCount();
13182	freedBlockSize = move.srcAllocation->GetBlock()->m_pMetadata->GetSize();
13183	}
13184	vector->Free(move.srcAllocation);
13185	{
13186	VmaMutexLockRead lock(vector->GetMutex(), vector->GetAllocator()->m_UseMutex);
13187	currentCount = vector->GetBlockCount();
13188	}
13189	freedBlockSize *= prevCount - currentCount;
13190
13191	VkDeviceSize dstBlockSize;
13192	{
13193	VmaMutexLockRead lock(vector->GetMutex(), vector->GetAllocator()->m_UseMutex);
13194	dstBlockSize = move.dstTmpAllocation->GetBlock()->m_pMetadata->GetSize();
13195	}
13196	vector->Free(move.dstTmpAllocation);
13197	{
13198	VmaMutexLockRead lock(vector->GetMutex(), vector->GetAllocator()->m_UseMutex);
13199	freedBlockSize += dstBlockSize * (currentCount - vector->GetBlockCount());
13200	currentCount = vector->GetBlockCount();
13201	}
13202
13203	result = VK_INCOMPLETE;
13204	break;
13205	}
13206	default:
13207	VMA_ASSERT(`0`);
13208	}
13209
13210	if (prevCount > currentCount)
13211	{
13212	size_t freedBlocks = prevCount - currentCount;
13213	m_PassStats.deviceMemoryBlocksFreed += static_cast<uint32_t>(freedBlocks);
13214	m_PassStats.bytesFreed += freedBlockSize;
13215	}
13216
13217	switch (m_Algorithm)
13218	{
13219	case VMA_DEFRAGMENTATION_FLAG_ALGORITHM_EXTENSIVE_BIT:
13220	{
13221	if (m_AlgorithmState != VMA_NULL)
13222	{
13223	// Avoid unnecessary tries to allocate when new free block is avaiable
13224	StateExtensive& state = reinterpret_cast<StateExtensive*>(m_AlgorithmState)[vectorIndex];
13225	if (state.firstFreeBlock != SIZE_MAX)
13226	{
13227	const size_t diff = prevCount - currentCount;
13228	if (state.firstFreeBlock >= diff)
13229	{
13230	state.firstFreeBlock -= diff;
13231	if (state.firstFreeBlock != `0`)
13232	state.firstFreeBlock -= vector->GetBlock(state.firstFreeBlock - `1`)->m_pMetadata->IsEmpty();
13233	}
13234	else
13235	state.firstFreeBlock = `0`;
13236	}
13237	}
13238	}
13239	}
13240	}
13241	moveInfo.moveCount = `0`;
13242	moveInfo.pMoves = VMA_NULL;
13243	m_Moves.clear();
13244
13245	// Update stats
13246	m_GlobalStats.allocationsMoved += m_PassStats.allocationsMoved;
13247	m_GlobalStats.bytesFreed += m_PassStats.bytesFreed;
13248	m_GlobalStats.bytesMoved += m_PassStats.bytesMoved;
13249	m_GlobalStats.deviceMemoryBlocksFreed += m_PassStats.deviceMemoryBlocksFreed;
13250	m_PassStats = { `0` };
13251
13252	// Move blocks with immovable allocations according to algorithm
13253	if (immovableBlocks.size() > `0`)
13254	{
13255	switch (m_Algorithm)
13256	{
13257	case VMA_DEFRAGMENTATION_FLAG_ALGORITHM_EXTENSIVE_BIT:
13258	{
13259	if (m_AlgorithmState != VMA_NULL)
13260	{
13261	bool swapped = false;
13262	// Move to the start of free blocks range
13263	for (const FragmentedBlock& block : immovableBlocks)
13264	{
13265	StateExtensive& state = reinterpret_cast<StateExtensive*>(m_AlgorithmState)[block.data];
13266	if (state.operation != StateExtensive::Operation::Cleanup)
13267	{
13268	VmaBlockVector* vector = m_pBlockVectors[block.data];
13269	VmaMutexLockWrite lock(vector->GetMutex(), vector->GetAllocator()->m_UseMutex);
13270
13271	for (size_t i = `0`, count = vector->GetBlockCount() - m_ImmovableBlockCount; i < count; ++i)
13272	{
13273	if (vector->GetBlock(i) == block.block)
13274	{
13275	VMA_SWAP(vector->m_Blocks[i], vector->m_Blocks[vector->GetBlockCount() - ++m_ImmovableBlockCount]);
13276	if (state.firstFreeBlock != SIZE_MAX)
13277	{
13278	if (i + `1` < state.firstFreeBlock)
13279	{
13280	if (state.firstFreeBlock > `1`)
13281	VMA_SWAP(vector->m_Blocks[i], vector->m_Blocks[--state.firstFreeBlock]);
13282	else
13283	--state.firstFreeBlock;
13284	}
13285	}
13286	swapped = true;
13287	break;
13288	}
13289	}
13290	}
13291	}
13292	if (swapped)
13293	result = VK_INCOMPLETE;
13294	break;
13295	}
13296	}
13297	default:
13298	{
13299	// Move to the begining
13300	for (const FragmentedBlock& block : immovableBlocks)
13301	{
13302	VmaBlockVector* vector = m_pBlockVectors[block.data];
13303	VmaMutexLockWrite lock(vector->GetMutex(), vector->GetAllocator()->m_UseMutex);
13304
13305	for (size_t i = m_ImmovableBlockCount; i < vector->GetBlockCount(); ++i)
13306	{
13307	if (vector->GetBlock(i) == block.block)
13308	{
13309	VMA_SWAP(vector->m_Blocks[i], vector->m_Blocks[m_ImmovableBlockCount++]);
13310	break;
13311	}
13312	}
13313	}
13314	break;
13315	}
13316	}
13317	}
13318
13319	// Bulk-map destination blocks
13320	for (const FragmentedBlock& block : mappedBlocks)
13321	{
13322	VkResult res = block.block->Map(allocator, block.data, VMA_NULL);
13323	VMA_ASSERT(res == VK_SUCCESS);
13324	}
13325	return result;
13326	}
13327
13328	bool VmaDefragmentationContext_T::ComputeDefragmentation(VmaBlockVector& vector, size_t index)
13329	{
13330	switch (m_Algorithm)
13331	{
13332	case VMA_DEFRAGMENTATION_FLAG_ALGORITHM_FAST_BIT:
13333	return ComputeDefragmentation_Fast(vector);
13334	default:
13335	VMA_ASSERT(`0`);
13336	case VMA_DEFRAGMENTATION_FLAG_ALGORITHM_BALANCED_BIT:
13337	return ComputeDefragmentation_Balanced(vector, index, true);
13338	case VMA_DEFRAGMENTATION_FLAG_ALGORITHM_FULL_BIT:
13339	return ComputeDefragmentation_Full(vector);
13340	case VMA_DEFRAGMENTATION_FLAG_ALGORITHM_EXTENSIVE_BIT:
13341	return ComputeDefragmentation_Extensive(vector, index);
13342	}
13343	}
13344
13345	VmaDefragmentationContext_T::MoveAllocationData VmaDefragmentationContext_T::GetMoveData(
13346	VmaAllocHandle handle, VmaBlockMetadata* metadata)
13347	{
13348	MoveAllocationData moveData;
13349	moveData.move.srcAllocation = (VmaAllocation)metadata->GetAllocationUserData(handle);
13350	moveData.size = moveData.move.srcAllocation->GetSize();
13351	moveData.alignment = moveData.move.srcAllocation->GetAlignment();
13352	moveData.type = moveData.move.srcAllocation->GetSuballocationType();
13353	moveData.flags = `0`;
13354
13355	if (moveData.move.srcAllocation->IsPersistentMap())
13356	moveData.flags \|= VMA_ALLOCATION_CREATE_MAPPED_BIT;
13357	if (moveData.move.srcAllocation->IsMappingAllowed())
13358	moveData.flags \|= VMA_ALLOCATION_CREATE_HOST_ACCESS_SEQUENTIAL_WRITE_BIT \| VMA_ALLOCATION_CREATE_HOST_ACCESS_RANDOM_BIT;
13359
13360	return moveData;
13361	}
13362
13363	VmaDefragmentationContext_T::CounterStatus VmaDefragmentationContext_T::CheckCounters(VkDeviceSize bytes)
13364	{
13365	// Ignore allocation if will exceed max size for copy
13366	if (m_PassStats.bytesMoved + bytes > m_MaxPassBytes)
13367	{
13368	if (++m_IgnoredAllocs < MAX_ALLOCS_TO_IGNORE)
13369	return CounterStatus::Ignore;
13370	else
13371	return CounterStatus::End;
13372	}
13373	return CounterStatus::Pass;
13374	}
13375
13376	bool VmaDefragmentationContext_T::IncrementCounters(VkDeviceSize bytes)
13377	{
13378	m_PassStats.bytesMoved += bytes;
13379	// Early return when max found
13380	if (++m_PassStats.allocationsMoved >= m_MaxPassAllocations \|\| m_PassStats.bytesMoved >= m_MaxPassBytes)
13381	{
13382	VMA_ASSERT(m_PassStats.allocationsMoved == m_MaxPassAllocations \|\|
13383	m_PassStats.bytesMoved == m_MaxPassBytes && "Exceeded maximal pass threshold!");
13384	return true;
13385	}
13386	return false;
13387	}
13388
13389	bool VmaDefragmentationContext_T::ReallocWithinBlock(VmaBlockVector& vector, VmaDeviceMemoryBlock* block)
13390	{
13391	VmaBlockMetadata* metadata = block->m_pMetadata;
13392
13393	for (VmaAllocHandle handle = metadata->GetAllocationListBegin();
13394	handle != VK_NULL_HANDLE;
13395	handle = metadata->GetNextAllocation(handle))
13396	{
13397	MoveAllocationData moveData = GetMoveData(handle, metadata);
13398	// Ignore newly created allocations by defragmentation algorithm
13399	if (moveData.move.srcAllocation->GetUserData() == this)
13400	continue;
13401	switch (CheckCounters(moveData.move.srcAllocation->GetSize()))
13402	{
13403	case CounterStatus::Ignore:
13404	continue;
13405	case CounterStatus::End:
13406	return true;
13407	default:
13408	VMA_ASSERT(`0`);
13409	case CounterStatus::Pass:
13410	break;
13411	}
13412
13413	VkDeviceSize offset = moveData.move.srcAllocation->GetOffset();
13414	if (offset != `0` && metadata->GetSumFreeSize() >= moveData.size)
13415	{
13416	VmaAllocationRequest request = {};
13417	if (metadata->CreateAllocationRequest(
13418	moveData.size,
13419	moveData.alignment,
13420	false,
13421	moveData.type,
13422	VMA_ALLOCATION_CREATE_STRATEGY_MIN_OFFSET_BIT,
13423	&request))
13424	{
13425	if (metadata->GetAllocationOffset(request.allocHandle) < offset)
13426	{
13427	if (vector.CommitAllocationRequest(
13428	request,
13429	block,
13430	moveData.alignment,
13431	moveData.flags,
13432	this,
13433	moveData.type,
13434	&moveData.move.dstTmpAllocation) == VK_SUCCESS)
13435	{
13436	m_Moves.push_back(moveData.move);
13437	if (IncrementCounters(moveData.size))
13438	return true;
13439	}
13440	}
13441	}
13442	}
13443	}
13444	return false;
13445	}
13446
13447	bool VmaDefragmentationContext_T::AllocInOtherBlock(size_t start, size_t end, MoveAllocationData& data, VmaBlockVector& vector)
13448	{
13449	for (; start < end; ++start)
13450	{
13451	VmaDeviceMemoryBlock* dstBlock = vector.GetBlock(start);
13452	if (dstBlock->m_pMetadata->GetSumFreeSize() >= data.size)
13453	{
13454	if (vector.AllocateFromBlock(dstBlock,
13455	data.size,
13456	data.alignment,
13457	data.flags,
13458	this,
13459	data.type,
13460	`0`,
13461	&data.move.dstTmpAllocation) == VK_SUCCESS)
13462	{
13463	m_Moves.push_back(data.move);
13464	if (IncrementCounters(data.size))
13465	return true;
13466	break;
13467	}
13468	}
13469	}
13470	return false;
13471	}
13472
13473	bool VmaDefragmentationContext_T::ComputeDefragmentation_Fast(VmaBlockVector& vector)
13474	{
13475	// Move only between blocks
13476
13477	// Go through allocations in last blocks and try to fit them inside first ones
13478	for (size_t i = vector.GetBlockCount() - `1`; i > m_ImmovableBlockCount; --i)
13479	{
13480	VmaBlockMetadata* metadata = vector.GetBlock(i)->m_pMetadata;
13481
13482	for (VmaAllocHandle handle = metadata->GetAllocationListBegin();
13483	handle != VK_NULL_HANDLE;
13484	handle = metadata->GetNextAllocation(handle))
13485	{
13486	MoveAllocationData moveData = GetMoveData(handle, metadata);
13487	// Ignore newly created allocations by defragmentation algorithm
13488	if (moveData.move.srcAllocation->GetUserData() == this)
13489	continue;
13490	switch (CheckCounters(moveData.move.srcAllocation->GetSize()))
13491	{
13492	case CounterStatus::Ignore:
13493	continue;
13494	case CounterStatus::End:
13495	return true;
13496	default:
13497	VMA_ASSERT(`0`);
13498	case CounterStatus::Pass:
13499	break;
13500	}
13501
13502	// Check all previous blocks for free space
13503	if (AllocInOtherBlock(`0`, i, moveData, vector))
13504	return true;
13505	}
13506	}
13507	return false;
13508	}
13509
13510	bool VmaDefragmentationContext_T::ComputeDefragmentation_Balanced(VmaBlockVector& vector, size_t index, bool update)
13511	{
13512	// Go over every allocation and try to fit it in previous blocks at lowest offsets,
13513	// if not possible: realloc within single block to minimize offset (exclude offset == 0),
13514	// but only if there are noticable gaps between them (some heuristic, ex. average size of allocation in block)
13515	VMA_ASSERT(m_AlgorithmState != VMA_NULL);
13516
13517	StateBalanced& vectorState = reinterpret_cast<StateBalanced*>(m_AlgorithmState)[index];
13518	if (update && vectorState.avgAllocSize == UINT64_MAX)
13519	UpdateVectorStatistics(vector, vectorState);
13520
13521	const size_t startMoveCount = m_Moves.size();
13522	VkDeviceSize minimalFreeRegion = vectorState.avgFreeSize / `2`;
13523	for (size_t i = vector.GetBlockCount() - `1`; i > m_ImmovableBlockCount; --i)
13524	{
13525	VmaDeviceMemoryBlock* block = vector.GetBlock(i);
13526	VmaBlockMetadata* metadata = block->m_pMetadata;
13527	VkDeviceSize prevFreeRegionSize = `0`;
13528
13529	for (VmaAllocHandle handle = metadata->GetAllocationListBegin();
13530	handle != VK_NULL_HANDLE;
13531	handle = metadata->GetNextAllocation(handle))
13532	{
13533	MoveAllocationData moveData = GetMoveData(handle, metadata);
13534	// Ignore newly created allocations by defragmentation algorithm
13535	if (moveData.move.srcAllocation->GetUserData() == this)
13536	continue;
13537	switch (CheckCounters(moveData.move.srcAllocation->GetSize()))
13538	{
13539	case CounterStatus::Ignore:
13540	continue;
13541	case CounterStatus::End:
13542	return true;
13543	default:
13544	VMA_ASSERT(`0`);
13545	case CounterStatus::Pass:
13546	break;
13547	}
13548
13549	// Check all previous blocks for free space
13550	const size_t prevMoveCount = m_Moves.size();
13551	if (AllocInOtherBlock(`0`, i, moveData, vector))
13552	return true;
13553
13554	VkDeviceSize nextFreeRegionSize = metadata->GetNextFreeRegionSize(handle);
13555	// If no room found then realloc within block for lower offset
13556	VkDeviceSize offset = moveData.move.srcAllocation->GetOffset();
13557	if (prevMoveCount == m_Moves.size() && offset != `0` && metadata->GetSumFreeSize() >= moveData.size)
13558	{
13559	// Check if realloc will make sense
13560	if (prevFreeRegionSize >= minimalFreeRegion \|\|
13561	nextFreeRegionSize >= minimalFreeRegion \|\|
13562	moveData.size <= vectorState.avgFreeSize \|\|
13563	moveData.size <= vectorState.avgAllocSize)
13564	{
13565	VmaAllocationRequest request = {};
13566	if (metadata->CreateAllocationRequest(
13567	moveData.size,
13568	moveData.alignment,
13569	false,
13570	moveData.type,
13571	VMA_ALLOCATION_CREATE_STRATEGY_MIN_OFFSET_BIT,
13572	&request))
13573	{
13574	if (metadata->GetAllocationOffset(request.allocHandle) < offset)
13575	{
13576	if (vector.CommitAllocationRequest(
13577	request,
13578	block,
13579	moveData.alignment,
13580	moveData.flags,
13581	this,
13582	moveData.type,
13583	&moveData.move.dstTmpAllocation) == VK_SUCCESS)
13584	{
13585	m_Moves.push_back(moveData.move);
13586	if (IncrementCounters(moveData.size))
13587	return true;
13588	}
13589	}
13590	}
13591	}
13592	}
13593	prevFreeRegionSize = nextFreeRegionSize;
13594	}
13595	}
13596
13597	// No moves perfomed, update statistics to current vector state
13598	if (startMoveCount == m_Moves.size() && !update)
13599	{
13600	vectorState.avgAllocSize = UINT64_MAX;
13601	return ComputeDefragmentation_Balanced(vector, index, false);
13602	}
13603	return false;
13604	}
13605
13606	bool VmaDefragmentationContext_T::ComputeDefragmentation_Full(VmaBlockVector& vector)
13607	{
13608	// Go over every allocation and try to fit it in previous blocks at lowest offsets,
13609	// if not possible: realloc within single block to minimize offset (exclude offset == 0)
13610
13611	for (size_t i = vector.GetBlockCount() - `1`; i > m_ImmovableBlockCount; --i)
13612	{
13613	VmaDeviceMemoryBlock* block = vector.GetBlock(i);
13614	VmaBlockMetadata* metadata = block->m_pMetadata;
13615
13616	for (VmaAllocHandle handle = metadata->GetAllocationListBegin();
13617	handle != VK_NULL_HANDLE;
13618	handle = metadata->GetNextAllocation(handle))
13619	{
13620	MoveAllocationData moveData = GetMoveData(handle, metadata);
13621	// Ignore newly created allocations by defragmentation algorithm
13622	if (moveData.move.srcAllocation->GetUserData() == this)
13623	continue;
13624	switch (CheckCounters(moveData.move.srcAllocation->GetSize()))
13625	{
13626	case CounterStatus::Ignore:
13627	continue;
13628	case CounterStatus::End:
13629	return true;
13630	default:
13631	VMA_ASSERT(`0`);
13632	case CounterStatus::Pass:
13633	break;
13634	}
13635
13636	// Check all previous blocks for free space
13637	const size_t prevMoveCount = m_Moves.size();
13638	if (AllocInOtherBlock(`0`, i, moveData, vector))
13639	return true;
13640
13641	// If no room found then realloc within block for lower offset
13642	VkDeviceSize offset = moveData.move.srcAllocation->GetOffset();
13643	if (prevMoveCount == m_Moves.size() && offset != `0` && metadata->GetSumFreeSize() >= moveData.size)
13644	{
13645	VmaAllocationRequest request = {};
13646	if (metadata->CreateAllocationRequest(
13647	moveData.size,
13648	moveData.alignment,
13649	false,
13650	moveData.type,
13651	VMA_ALLOCATION_CREATE_STRATEGY_MIN_OFFSET_BIT,
13652	&request))
13653	{
13654	if (metadata->GetAllocationOffset(request.allocHandle) < offset)
13655	{
13656	if (vector.CommitAllocationRequest(
13657	request,
13658	block,
13659	moveData.alignment,
13660	moveData.flags,
13661	this,
13662	moveData.type,
13663	&moveData.move.dstTmpAllocation) == VK_SUCCESS)
13664	{
13665	m_Moves.push_back(moveData.move);
13666	if (IncrementCounters(moveData.size))
13667	return true;
13668	}
13669	}
13670	}
13671	}
13672	}
13673	}
13674	return false;
13675	}
13676
13677	bool VmaDefragmentationContext_T::ComputeDefragmentation_Extensive(VmaBlockVector& vector, size_t index)
13678	{
13679	// First free single block, then populate it to the brim, then free another block, and so on
13680
13681	// Fallback to previous algorithm since without granularity conflicts it can achieve max packing
13682	if (vector.m_BufferImageGranularity == `1`)
13683	return ComputeDefragmentation_Full(vector);
13684
13685	VMA_ASSERT(m_AlgorithmState != VMA_NULL);
13686
13687	StateExtensive& vectorState = reinterpret_cast<StateExtensive*>(m_AlgorithmState)[index];
13688
13689	bool texturePresent = false, bufferPresent = false, otherPresent = false;
13690	switch (vectorState.operation)
13691	{
13692	case StateExtensive::Operation::Done: // Vector defragmented
13693	return false;
13694	case StateExtensive::Operation::FindFreeBlockBuffer:
13695	case StateExtensive::Operation::FindFreeBlockTexture:
13696	case StateExtensive::Operation::FindFreeBlockAll:
13697	{
13698	// No more blocks to free, just perform fast realloc and move to cleanup
13699	if (vectorState.firstFreeBlock == `0`)
13700	{
13701	vectorState.operation = StateExtensive::Operation::Cleanup;
13702	return ComputeDefragmentation_Fast(vector);
13703	}
13704
13705	// No free blocks, have to clear last one
13706	size_t last = (vectorState.firstFreeBlock == SIZE_MAX ? vector.GetBlockCount() : vectorState.firstFreeBlock) - `1`;
13707	VmaBlockMetadata* freeMetadata = vector.GetBlock(last)->m_pMetadata;
13708
13709	const size_t prevMoveCount = m_Moves.size();
13710	for (VmaAllocHandle handle = freeMetadata->GetAllocationListBegin();
13711	handle != VK_NULL_HANDLE;
13712	handle = freeMetadata->GetNextAllocation(handle))
13713	{
13714	MoveAllocationData moveData = GetMoveData(handle, freeMetadata);
13715	switch (CheckCounters(moveData.move.srcAllocation->GetSize()))
13716	{
13717	case CounterStatus::Ignore:
13718	continue;
13719	case CounterStatus::End:
13720	return true;
13721	default:
13722	VMA_ASSERT(`0`);
13723	case CounterStatus::Pass:
13724	break;
13725	}
13726
13727	// Check all previous blocks for free space
13728	if (AllocInOtherBlock(`0`, last, moveData, vector))
13729	{
13730	// Full clear performed already
13731	if (prevMoveCount != m_Moves.size() && freeMetadata->GetNextAllocation(handle) == VK_NULL_HANDLE)
13732	reinterpret_cast<size_t*>(m_AlgorithmState)[index] = last;
13733	return true;
13734	}
13735	}
13736
13737	if (prevMoveCount == m_Moves.size())
13738	{
13739	// Cannot perform full clear, have to move data in other blocks around
13740	if (last != `0`)
13741	{
13742	for (size_t i = last - `1`; i; --i)
13743	{
13744	if (ReallocWithinBlock(vector, vector.GetBlock(i)))
13745	return true;
13746	}
13747	}
13748
13749	if (prevMoveCount == m_Moves.size())
13750	{
13751	// No possible reallocs within blocks, try to move them around fast
13752	return ComputeDefragmentation_Fast(vector);
13753	}
13754	}
13755	else
13756	{
13757	switch (vectorState.operation)
13758	{
13759	case StateExtensive::Operation::FindFreeBlockBuffer:
13760	vectorState.operation = StateExtensive::Operation::MoveBuffers;
13761	break;
13762	default:
13763	VMA_ASSERT(`0`);
13764	case StateExtensive::Operation::FindFreeBlockTexture:
13765	vectorState.operation = StateExtensive::Operation::MoveTextures;
13766	break;
13767	case StateExtensive::Operation::FindFreeBlockAll:
13768	vectorState.operation = StateExtensive::Operation::MoveAll;
13769	break;
13770	}
13771	vectorState.firstFreeBlock = last;
13772	// Nothing done, block found without reallocations, can perform another reallocs in same pass
13773	return ComputeDefragmentation_Extensive(vector, index);
13774	}
13775	break;
13776	}
13777	case StateExtensive::Operation::MoveTextures:
13778	{
13779	if (MoveDataToFreeBlocks(VMA_SUBALLOCATION_TYPE_IMAGE_OPTIMAL, vector,
13780	vectorState.firstFreeBlock, texturePresent, bufferPresent, otherPresent))
13781	{
13782	if (texturePresent)
13783	{
13784	vectorState.operation = StateExtensive::Operation::FindFreeBlockTexture;
13785	return ComputeDefragmentation_Extensive(vector, index);
13786	}
13787
13788	if (!bufferPresent && !otherPresent)
13789	{
13790	vectorState.operation = StateExtensive::Operation::Cleanup;
13791	break;
13792	}
13793
13794	// No more textures to move, check buffers
13795	vectorState.operation = StateExtensive::Operation::MoveBuffers;
13796	bufferPresent = false;
13797	otherPresent = false;
13798	}
13799	else
13800	break;
13801	}
13802	case StateExtensive::Operation::MoveBuffers:
13803	{
13804	if (MoveDataToFreeBlocks(VMA_SUBALLOCATION_TYPE_BUFFER, vector,
13805	vectorState.firstFreeBlock, texturePresent, bufferPresent, otherPresent))
13806	{
13807	if (bufferPresent)
13808	{
13809	vectorState.operation = StateExtensive::Operation::FindFreeBlockBuffer;
13810	return ComputeDefragmentation_Extensive(vector, index);
13811	}
13812
13813	if (!otherPresent)
13814	{
13815	vectorState.operation = StateExtensive::Operation::Cleanup;
13816	break;
13817	}
13818
13819	// No more buffers to move, check all others
13820	vectorState.operation = StateExtensive::Operation::MoveAll;
13821	otherPresent = false;
13822	}
13823	else
13824	break;
13825	}
13826	case StateExtensive::Operation::MoveAll:
13827	{
13828	if (MoveDataToFreeBlocks(VMA_SUBALLOCATION_TYPE_FREE, vector,
13829	vectorState.firstFreeBlock, texturePresent, bufferPresent, otherPresent))
13830	{
13831	if (otherPresent)
13832	{
13833	vectorState.operation = StateExtensive::Operation::FindFreeBlockBuffer;
13834	return ComputeDefragmentation_Extensive(vector, index);
13835	}
13836	// Everything moved
13837	vectorState.operation = StateExtensive::Operation::Cleanup;
13838	}
13839	break;
13840	}
13841	case StateExtensive::Operation::Cleanup:
13842	// Cleanup is handled below so that other operations may reuse the cleanup code. This case is here to prevent the unhandled enum value warning (C4062).
13843	break;
13844	}
13845
13846	if (vectorState.operation == StateExtensive::Operation::Cleanup)
13847	{
13848	// All other work done, pack data in blocks even tighter if possible
13849	const size_t prevMoveCount = m_Moves.size();
13850	for (size_t i = `0`; i < vector.GetBlockCount(); ++i)
13851	{
13852	if (ReallocWithinBlock(vector, vector.GetBlock(i)))
13853	return true;
13854	}
13855
13856	if (prevMoveCount == m_Moves.size())
13857	vectorState.operation = StateExtensive::Operation::Done;
13858	}
13859	return false;
13860	}
13861
13862	void VmaDefragmentationContext_T::UpdateVectorStatistics(VmaBlockVector& vector, StateBalanced& state)
13863	{
13864	size_t allocCount = `0`;
13865	size_t freeCount = `0`;
13866	state.avgFreeSize = `0`;
13867	state.avgAllocSize = `0`;
13868
13869	for (size_t i = `0`; i < vector.GetBlockCount(); ++i)
13870	{
13871	VmaBlockMetadata* metadata = vector.GetBlock(i)->m_pMetadata;
13872
13873	allocCount += metadata->GetAllocationCount();
13874	freeCount += metadata->GetFreeRegionsCount();
13875	state.avgFreeSize += metadata->GetSumFreeSize();
13876	state.avgAllocSize += metadata->GetSize();
13877	}
13878
13879	state.avgAllocSize = (state.avgAllocSize - state.avgFreeSize) / allocCount;
13880	state.avgFreeSize /= freeCount;
13881	}
13882
13883	bool VmaDefragmentationContext_T::MoveDataToFreeBlocks(VmaSuballocationType currentType,
13884	VmaBlockVector& vector, size_t firstFreeBlock,
13885	bool& texturePresent, bool& bufferPresent, bool& otherPresent)
13886	{
13887	const size_t prevMoveCount = m_Moves.size();
13888	for (size_t i = firstFreeBlock ; i;)
13889	{
13890	VmaDeviceMemoryBlock* block = vector.GetBlock(--i);
13891	VmaBlockMetadata* metadata = block->m_pMetadata;
13892
13893	for (VmaAllocHandle handle = metadata->GetAllocationListBegin();
13894	handle != VK_NULL_HANDLE;
13895	handle = metadata->GetNextAllocation(handle))
13896	{
13897	MoveAllocationData moveData = GetMoveData(handle, metadata);
13898	// Ignore newly created allocations by defragmentation algorithm
13899	if (moveData.move.srcAllocation->GetUserData() == this)
13900	continue;
13901	switch (CheckCounters(moveData.move.srcAllocation->GetSize()))
13902	{
13903	case CounterStatus::Ignore:
13904	continue;
13905	case CounterStatus::End:
13906	return true;
13907	default:
13908	VMA_ASSERT(`0`);
13909	case CounterStatus::Pass:
13910	break;
13911	}
13912
13913	// Move only single type of resources at once
13914	if (!VmaIsBufferImageGranularityConflict(moveData.type, currentType))
13915	{
13916	// Try to fit allocation into free blocks
13917	if (AllocInOtherBlock(firstFreeBlock, vector.GetBlockCount(), moveData, vector))
13918	return false;
13919	}
13920
13921	if (!VmaIsBufferImageGranularityConflict(moveData.type, VMA_SUBALLOCATION_TYPE_IMAGE_OPTIMAL))
13922	texturePresent = true;
13923	else if (!VmaIsBufferImageGranularityConflict(moveData.type, VMA_SUBALLOCATION_TYPE_BUFFER))
13924	bufferPresent = true;
13925	else
13926	otherPresent = true;
13927	}
13928	}
13929	return prevMoveCount == m_Moves.size();
13930	}
13931	#endif // _VMA_DEFRAGMENTATION_CONTEXT_FUNCTIONS
13932
13933	#ifndef _VMA_POOL_T_FUNCTIONS
13934	VmaPool_T::VmaPool_T(
13935	VmaAllocator hAllocator,
13936	const VmaPoolCreateInfo& createInfo,
13937	VkDeviceSize preferredBlockSize)
13938	: m_BlockVector(
13939	hAllocator,
13940	this, // hParentPool
13941	createInfo.memoryTypeIndex,
13942	createInfo.blockSize != `0` ? createInfo.blockSize : preferredBlockSize,
13943	createInfo.minBlockCount,
13944	createInfo.maxBlockCount,
13945	(createInfo.flags& VMA_POOL_CREATE_IGNORE_BUFFER_IMAGE_GRANULARITY_BIT) != `0` ? `1` : hAllocator->GetBufferImageGranularity(),
13946	createInfo.blockSize != `0`, // explicitBlockSize
13947	createInfo.flags & VMA_POOL_CREATE_ALGORITHM_MASK, // algorithm
13948	createInfo.priority,
13949	VMA_MAX(hAllocator->GetMemoryTypeMinAlignment(createInfo.memoryTypeIndex), createInfo.minAllocationAlignment),
13950	createInfo.pMemoryAllocateNext),
13951	m_Id(`0`),
13952	m_Name(VMA_NULL) {}
13953
13954	VmaPool_T::~VmaPool_T()
13955	{
13956	VMA_ASSERT(m_PrevPool == VMA_NULL && m_NextPool == VMA_NULL);
13957	}
13958
13959	void VmaPool_T::SetName(const char* pName)
13960	{
13961	const VkAllocationCallbacks* allocs = m_BlockVector.GetAllocator()->GetAllocationCallbacks();
13962	VmaFreeString(allocs, m_Name);
13963
13964	if (pName != VMA_NULL)
13965	{
13966	m_Name = VmaCreateStringCopy(allocs, pName);
13967	}
13968	else
13969	{
13970	m_Name = VMA_NULL;
13971	}
13972	}
13973	#endif // _VMA_POOL_T_FUNCTIONS
13974
13975	#ifndef _VMA_ALLOCATOR_T_FUNCTIONS
13976	VmaAllocator_T::VmaAllocator_T(const VmaAllocatorCreateInfo* pCreateInfo) :
13977	m_UseMutex((pCreateInfo->flags & VMA_ALLOCATOR_CREATE_EXTERNALLY_SYNCHRONIZED_BIT) == `0`),
13978	m_VulkanApiVersion(pCreateInfo->vulkanApiVersion != `0` ? pCreateInfo->vulkanApiVersion : VK_API_VERSION_1_0),
13979	m_UseKhrDedicatedAllocation((pCreateInfo->flags & VMA_ALLOCATOR_CREATE_KHR_DEDICATED_ALLOCATION_BIT) != `0`),
13980	m_UseKhrBindMemory2((pCreateInfo->flags & VMA_ALLOCATOR_CREATE_KHR_BIND_MEMORY2_BIT) != `0`),
13981	m_UseExtMemoryBudget((pCreateInfo->flags & VMA_ALLOCATOR_CREATE_EXT_MEMORY_BUDGET_BIT) != `0`),
13982	m_UseAmdDeviceCoherentMemory((pCreateInfo->flags & VMA_ALLOCATOR_CREATE_AMD_DEVICE_COHERENT_MEMORY_BIT) != `0`),
13983	m_UseKhrBufferDeviceAddress((pCreateInfo->flags & VMA_ALLOCATOR_CREATE_BUFFER_DEVICE_ADDRESS_BIT) != `0`),
13984	m_UseExtMemoryPriority((pCreateInfo->flags & VMA_ALLOCATOR_CREATE_EXT_MEMORY_PRIORITY_BIT) != `0`),
13985	m_hDevice(pCreateInfo->device),
13986	m_hInstance(pCreateInfo->instance),
13987	m_AllocationCallbacksSpecified(pCreateInfo->pAllocationCallbacks != VMA_NULL),
13988	m_AllocationCallbacks(pCreateInfo->pAllocationCallbacks ?
13989	*pCreateInfo->pAllocationCallbacks : VmaEmptyAllocationCallbacks),
13990	m_AllocationObjectAllocator(&m_AllocationCallbacks),
13991	m_HeapSizeLimitMask(`0`),
13992	m_DeviceMemoryCount(`0`),
13993	m_PreferredLargeHeapBlockSize(`0`),
13994	m_PhysicalDevice(pCreateInfo->physicalDevice),
13995	m_GpuDefragmentationMemoryTypeBits(UINT32_MAX),
13996	m_NextPoolId(`0`),
13997	m_GlobalMemoryTypeBits(UINT32_MAX)
13998	{
13999	if(m_VulkanApiVersion >= VK_MAKE_VERSION(`1`, `1`, `0`))
14000	{
14001	m_UseKhrDedicatedAllocation = false;
14002	m_UseKhrBindMemory2 = false;
14003	}
14004
14005	if(VMA_DEBUG_DETECT_CORRUPTION)
14006	{
14007	// Needs to be multiply of uint32_t size because we are going to write VMA_CORRUPTION_DETECTION_MAGIC_VALUE to it.
14008	VMA_ASSERT(VMA_DEBUG_MARGIN % sizeof(uint32_t) == `0`);
14009	}
14010
14011	VMA_ASSERT(pCreateInfo->physicalDevice && pCreateInfo->device && pCreateInfo->instance);
14012
14013	if(m_VulkanApiVersion < VK_MAKE_VERSION(`1`, `1`, `0`))
14014	{
14015	#if !(VMA_DEDICATED_ALLOCATION)
14016	if((pCreateInfo->flags & VMA_ALLOCATOR_CREATE_KHR_DEDICATED_ALLOCATION_BIT) != `0`)
14017	{
14018	VMA_ASSERT(`0` && "VMA_ALLOCATOR_CREATE_KHR_DEDICATED_ALLOCATION_BIT set but required extensions are disabled by preprocessor macros.");
14019	}
14020	#endif
14021	#if !(VMA_BIND_MEMORY2)
14022	if((pCreateInfo->flags & VMA_ALLOCATOR_CREATE_KHR_BIND_MEMORY2_BIT) != `0`)
14023	{
14024	VMA_ASSERT(`0` && "VMA_ALLOCATOR_CREATE_KHR_BIND_MEMORY2_BIT set but required extension is disabled by preprocessor macros.");
14025	}
14026	#endif
14027	}
14028	#if !(VMA_MEMORY_BUDGET)
14029	if((pCreateInfo->flags & VMA_ALLOCATOR_CREATE_EXT_MEMORY_BUDGET_BIT) != `0`)
14030	{
14031	VMA_ASSERT(`0` && "VMA_ALLOCATOR_CREATE_EXT_MEMORY_BUDGET_BIT set but required extension is disabled by preprocessor macros.");
14032	}
14033	#endif
14034	#if !(VMA_BUFFER_DEVICE_ADDRESS)
14035	if(m_UseKhrBufferDeviceAddress)
14036	{
14037	VMA_ASSERT(`0` && "VMA_ALLOCATOR_CREATE_BUFFER_DEVICE_ADDRESS_BIT is set but required extension or Vulkan 1.2 is not available in your Vulkan header or its support in VMA has been disabled by a preprocessor macro.");
14038	}
14039	#endif
14040	#if VMA_VULKAN_VERSION < 1002000
14041	if(m_VulkanApiVersion >= VK_MAKE_VERSION(`1`, `2`, `0`))
14042	{
14043	VMA_ASSERT(`0` && "vulkanApiVersion >= VK_API_VERSION_1_2 but required Vulkan version is disabled by preprocessor macros.");
14044	}
14045	#endif
14046	#if VMA_VULKAN_VERSION < 1001000
14047	if(m_VulkanApiVersion >= VK_MAKE_VERSION(`1`, `1`, `0`))
14048	{
14049	VMA_ASSERT(`0` && "vulkanApiVersion >= VK_API_VERSION_1_1 but required Vulkan version is disabled by preprocessor macros.");
14050	}
14051	#endif
14052	#if !(VMA_MEMORY_PRIORITY)
14053	if(m_UseExtMemoryPriority)
14054	{
14055	VMA_ASSERT(`0` && "VMA_ALLOCATOR_CREATE_EXT_MEMORY_PRIORITY_BIT is set but required extension is not available in your Vulkan header or its support in VMA has been disabled by a preprocessor macro.");
14056	}
14057	#endif
14058
14059	memset(&m_DeviceMemoryCallbacks, `0` ,sizeof(m_DeviceMemoryCallbacks));
14060	memset(&m_PhysicalDeviceProperties, `0`, sizeof(m_PhysicalDeviceProperties));
14061	memset(&m_MemProps, `0`, sizeof(m_MemProps));
14062
14063	memset(&m_pBlockVectors, `0`, sizeof(m_pBlockVectors));
14064	memset(&m_VulkanFunctions, `0`, sizeof(m_VulkanFunctions));
14065
14066	#if VMA_EXTERNAL_MEMORY
14067	memset(&m_TypeExternalMemoryHandleTypes, `0`, sizeof(m_TypeExternalMemoryHandleTypes));
14068	#endif // #if VMA_EXTERNAL_MEMORY
14069
14070	if(pCreateInfo->pDeviceMemoryCallbacks != VMA_NULL)
14071	{
14072	m_DeviceMemoryCallbacks.pUserData = pCreateInfo->pDeviceMemoryCallbacks->pUserData;
14073	m_DeviceMemoryCallbacks.pfnAllocate = pCreateInfo->pDeviceMemoryCallbacks->pfnAllocate;
14074	m_DeviceMemoryCallbacks.pfnFree = pCreateInfo->pDeviceMemoryCallbacks->pfnFree;
14075	}
14076
14077	ImportVulkanFunctions(pCreateInfo->pVulkanFunctions);
14078
14079	(*m_VulkanFunctions.vkGetPhysicalDeviceProperties)(m_PhysicalDevice, &m_PhysicalDeviceProperties);
14080	(*m_VulkanFunctions.vkGetPhysicalDeviceMemoryProperties)(m_PhysicalDevice, &m_MemProps);
14081
14082	VMA_ASSERT(VmaIsPow2(VMA_MIN_ALIGNMENT));
14083	VMA_ASSERT(VmaIsPow2(VMA_DEBUG_MIN_BUFFER_IMAGE_GRANULARITY));
14084	VMA_ASSERT(VmaIsPow2(m_PhysicalDeviceProperties.limits.bufferImageGranularity));
14085	VMA_ASSERT(VmaIsPow2(m_PhysicalDeviceProperties.limits.nonCoherentAtomSize));
14086
14087	m_PreferredLargeHeapBlockSize = (pCreateInfo->preferredLargeHeapBlockSize != `0`) ?
14088	pCreateInfo->preferredLargeHeapBlockSize : static_cast<VkDeviceSize>(VMA_DEFAULT_LARGE_HEAP_BLOCK_SIZE);
14089
14090	m_GlobalMemoryTypeBits = CalculateGlobalMemoryTypeBits();
14091
14092	#if VMA_EXTERNAL_MEMORY
14093	if(pCreateInfo->pTypeExternalMemoryHandleTypes != VMA_NULL)
14094	{
14095	memcpy(m_TypeExternalMemoryHandleTypes, pCreateInfo->pTypeExternalMemoryHandleTypes,
14096	sizeof(VkExternalMemoryHandleTypeFlagsKHR) * GetMemoryTypeCount());
14097	}
14098	#endif // #if VMA_EXTERNAL_MEMORY
14099
14100	if(pCreateInfo->pHeapSizeLimit != VMA_NULL)
14101	{
14102	for(uint32_t heapIndex = `0`; heapIndex < GetMemoryHeapCount(); ++heapIndex)
14103	{
14104	const VkDeviceSize limit = pCreateInfo->pHeapSizeLimit[heapIndex];
14105	if(limit != VK_WHOLE_SIZE)
14106	{
14107	m_HeapSizeLimitMask \|= `1u` << heapIndex;
14108	if(limit < m_MemProps.memoryHeaps[heapIndex].size)
14109	{
14110	m_MemProps.memoryHeaps[heapIndex].size = limit;
14111	}
14112	}
14113	}
14114	}
14115
14116	for(uint32_t memTypeIndex = `0`; memTypeIndex < GetMemoryTypeCount(); ++memTypeIndex)
14117	{
14118	// Create only supported types
14119	if((m_GlobalMemoryTypeBits & (`1u` << memTypeIndex)) != `0`)
14120	{
14121	const VkDeviceSize preferredBlockSize = CalcPreferredBlockSize(memTypeIndex);
14122	m_pBlockVectors[memTypeIndex] = vma_new(this, VmaBlockVector)(
14123	this,
14124	VK_NULL_HANDLE, // hParentPool
14125	memTypeIndex,
14126	preferredBlockSize,
14127	`0`,
14128	SIZE_MAX,
14129	GetBufferImageGranularity(),
14130	false, // explicitBlockSize
14131	`0`, // algorithm
14132	`0.5f`, // priority (0.5 is the default per Vulkan spec)
14133	GetMemoryTypeMinAlignment(memTypeIndex), // minAllocationAlignment
14134	VMA_NULL); // // pMemoryAllocateNext
14135	// No need to call m_pBlockVectors[memTypeIndex][blockVectorTypeIndex]->CreateMinBlocks here,
14136	// becase minBlockCount is 0.
14137	}
14138	}
14139	}
14140
14141	VkResult VmaAllocator_T::Init(const VmaAllocatorCreateInfo* pCreateInfo)
14142	{
14143	VkResult res = VK_SUCCESS;
14144
14145	#if VMA_MEMORY_BUDGET
14146	if(m_UseExtMemoryBudget)
14147	{
14148	UpdateVulkanBudget();
14149	}
14150	#endif // #if VMA_MEMORY_BUDGET
14151
14152	return res;
14153	}
14154
14155	VmaAllocator_T::~VmaAllocator_T()
14156	{
14157	VMA_ASSERT(m_Pools.IsEmpty());
14158
14159	for(size_t memTypeIndex = GetMemoryTypeCount(); memTypeIndex--; )
14160	{
14161	vma_delete(this, m_pBlockVectors[memTypeIndex]);
14162	}
14163	}
14164
14165	void VmaAllocator_T::ImportVulkanFunctions(const VmaVulkanFunctions* pVulkanFunctions)
14166	{
14167	#if VMA_STATIC_VULKAN_FUNCTIONS == 1
14168	ImportVulkanFunctions_Static();
14169	#endif
14170
14171	if(pVulkanFunctions != VMA_NULL)
14172	{
14173	ImportVulkanFunctions_Custom(pVulkanFunctions);
14174	}
14175
14176	#if VMA_DYNAMIC_VULKAN_FUNCTIONS == 1
14177	ImportVulkanFunctions_Dynamic();
14178	#endif
14179
14180	ValidateVulkanFunctions();
14181	}
14182
14183	#if VMA_STATIC_VULKAN_FUNCTIONS == 1
14184
14185	void VmaAllocator_T::ImportVulkanFunctions_Static()
14186	{
14187	// Vulkan 1.0
14188	m_VulkanFunctions.vkGetInstanceProcAddr = (PFN_vkGetInstanceProcAddr)vkGetInstanceProcAddr;
14189	m_VulkanFunctions.vkGetDeviceProcAddr = (PFN_vkGetDeviceProcAddr)vkGetDeviceProcAddr;
14190	m_VulkanFunctions.vkGetPhysicalDeviceProperties = (PFN_vkGetPhysicalDeviceProperties)vkGetPhysicalDeviceProperties;
14191	m_VulkanFunctions.vkGetPhysicalDeviceMemoryProperties = (PFN_vkGetPhysicalDeviceMemoryProperties)vkGetPhysicalDeviceMemoryProperties;
14192	m_VulkanFunctions.vkAllocateMemory = (PFN_vkAllocateMemory)vkAllocateMemory;
14193	m_VulkanFunctions.vkFreeMemory = (PFN_vkFreeMemory)vkFreeMemory;
14194	m_VulkanFunctions.vkMapMemory = (PFN_vkMapMemory)vkMapMemory;
14195	m_VulkanFunctions.vkUnmapMemory = (PFN_vkUnmapMemory)vkUnmapMemory;
14196	m_VulkanFunctions.vkFlushMappedMemoryRanges = (PFN_vkFlushMappedMemoryRanges)vkFlushMappedMemoryRanges;
14197	m_VulkanFunctions.vkInvalidateMappedMemoryRanges = (PFN_vkInvalidateMappedMemoryRanges)vkInvalidateMappedMemoryRanges;
14198	m_VulkanFunctions.vkBindBufferMemory = (PFN_vkBindBufferMemory)vkBindBufferMemory;
14199	m_VulkanFunctions.vkBindImageMemory = (PFN_vkBindImageMemory)vkBindImageMemory;
14200	m_VulkanFunctions.vkGetBufferMemoryRequirements = (PFN_vkGetBufferMemoryRequirements)vkGetBufferMemoryRequirements;
14201	m_VulkanFunctions.vkGetImageMemoryRequirements = (PFN_vkGetImageMemoryRequirements)vkGetImageMemoryRequirements;
14202	m_VulkanFunctions.vkCreateBuffer = (PFN_vkCreateBuffer)vkCreateBuffer;
14203	m_VulkanFunctions.vkDestroyBuffer = (PFN_vkDestroyBuffer)vkDestroyBuffer;
14204	m_VulkanFunctions.vkCreateImage = (PFN_vkCreateImage)vkCreateImage;
14205	m_VulkanFunctions.vkDestroyImage = (PFN_vkDestroyImage)vkDestroyImage;
14206	m_VulkanFunctions.vkCmdCopyBuffer = (PFN_vkCmdCopyBuffer)vkCmdCopyBuffer;
14207
14208	// Vulkan 1.1
14209	#if VMA_VULKAN_VERSION >= 1001000
14210	if(m_VulkanApiVersion >= VK_MAKE_VERSION(`1`, `1`, `0`))
14211	{
14212	m_VulkanFunctions.vkGetBufferMemoryRequirements2KHR = (PFN_vkGetBufferMemoryRequirements2)vkGetBufferMemoryRequirements2;
14213	m_VulkanFunctions.vkGetImageMemoryRequirements2KHR = (PFN_vkGetImageMemoryRequirements2)vkGetImageMemoryRequirements2;
14214	m_VulkanFunctions.vkBindBufferMemory2KHR = (PFN_vkBindBufferMemory2)vkBindBufferMemory2;
14215	m_VulkanFunctions.vkBindImageMemory2KHR = (PFN_vkBindImageMemory2)vkBindImageMemory2;
14216	m_VulkanFunctions.vkGetPhysicalDeviceMemoryProperties2KHR = (PFN_vkGetPhysicalDeviceMemoryProperties2)vkGetPhysicalDeviceMemoryProperties2;
14217	}
14218	#endif
14219
14220	#if VMA_VULKAN_VERSION >= 1003000
14221	if(m_VulkanApiVersion >= VK_MAKE_VERSION(`1`, `3`, `0`))
14222	{
14223	m_VulkanFunctions.vkGetDeviceBufferMemoryRequirements = (PFN_vkGetDeviceBufferMemoryRequirements)vkGetDeviceBufferMemoryRequirements;
14224	m_VulkanFunctions.vkGetDeviceImageMemoryRequirements = (PFN_vkGetDeviceImageMemoryRequirements)vkGetDeviceImageMemoryRequirements;
14225	}
14226	#endif
14227	}
14228
14229	#endif // VMA_STATIC_VULKAN_FUNCTIONS == 1
14230
14231	void VmaAllocator_T::ImportVulkanFunctions_Custom(const VmaVulkanFunctions* pVulkanFunctions)
14232	{
14233	VMA_ASSERT(pVulkanFunctions != VMA_NULL);
14234
14235	#define VMA_COPY_IF_NOT_NULL(funcName) \
14236	if(pVulkanFunctions->funcName != VMA_NULL) m_VulkanFunctions.funcName = pVulkanFunctions->funcName;
14237
14238	VMA_COPY_IF_NOT_NULL(vkGetInstanceProcAddr);
14239	VMA_COPY_IF_NOT_NULL(vkGetDeviceProcAddr);
14240	VMA_COPY_IF_NOT_NULL(vkGetPhysicalDeviceProperties);
14241	VMA_COPY_IF_NOT_NULL(vkGetPhysicalDeviceMemoryProperties);
14242	VMA_COPY_IF_NOT_NULL(vkAllocateMemory);
14243	VMA_COPY_IF_NOT_NULL(vkFreeMemory);
14244	VMA_COPY_IF_NOT_NULL(vkMapMemory);
14245	VMA_COPY_IF_NOT_NULL(vkUnmapMemory);
14246	VMA_COPY_IF_NOT_NULL(vkFlushMappedMemoryRanges);
14247	VMA_COPY_IF_NOT_NULL(vkInvalidateMappedMemoryRanges);
14248	VMA_COPY_IF_NOT_NULL(vkBindBufferMemory);
14249	VMA_COPY_IF_NOT_NULL(vkBindImageMemory);
14250	VMA_COPY_IF_NOT_NULL(vkGetBufferMemoryRequirements);
14251	VMA_COPY_IF_NOT_NULL(vkGetImageMemoryRequirements);
14252	VMA_COPY_IF_NOT_NULL(vkCreateBuffer);
14253	VMA_COPY_IF_NOT_NULL(vkDestroyBuffer);
14254	VMA_COPY_IF_NOT_NULL(vkCreateImage);
14255	VMA_COPY_IF_NOT_NULL(vkDestroyImage);
14256	VMA_COPY_IF_NOT_NULL(vkCmdCopyBuffer);
14257
14258	#if VMA_DEDICATED_ALLOCATION \|\| VMA_VULKAN_VERSION >= 1001000
14259	VMA_COPY_IF_NOT_NULL(vkGetBufferMemoryRequirements2KHR);
14260	VMA_COPY_IF_NOT_NULL(vkGetImageMemoryRequirements2KHR);
14261	#endif
14262
14263	#if VMA_BIND_MEMORY2 \|\| VMA_VULKAN_VERSION >= 1001000
14264	VMA_COPY_IF_NOT_NULL(vkBindBufferMemory2KHR);
14265	VMA_COPY_IF_NOT_NULL(vkBindImageMemory2KHR);
14266	#endif
14267
14268	#if VMA_MEMORY_BUDGET
14269	VMA_COPY_IF_NOT_NULL(vkGetPhysicalDeviceMemoryProperties2KHR);
14270	#endif
14271
14272	#if VMA_VULKAN_VERSION >= 1003000
14273	VMA_COPY_IF_NOT_NULL(vkGetDeviceBufferMemoryRequirements);
14274	VMA_COPY_IF_NOT_NULL(vkGetDeviceImageMemoryRequirements);
14275	#endif
14276
14277	#undef VMA_COPY_IF_NOT_NULL
14278	}
14279
14280	#if VMA_DYNAMIC_VULKAN_FUNCTIONS == 1
14281
14282	void VmaAllocator_T::ImportVulkanFunctions_Dynamic()
14283	{
14284	VMA_ASSERT(m_VulkanFunctions.vkGetInstanceProcAddr && m_VulkanFunctions.vkGetDeviceProcAddr &&
14285	"To use VMA_DYNAMIC_VULKAN_FUNCTIONS in new versions of VMA you now have to pass "
14286	"VmaVulkanFunctions::vkGetInstanceProcAddr and vkGetDeviceProcAddr as VmaAllocatorCreateInfo::pVulkanFunctions. "
14287	"Other members can be null.");
14288
14289	#define VMA_FETCH_INSTANCE_FUNC(memberName, functionPointerType, functionNameString) \
14290	if(m_VulkanFunctions.memberName == VMA_NULL) \
14291	m_VulkanFunctions.memberName = \
14292	(functionPointerType)m_VulkanFunctions.vkGetInstanceProcAddr(m_hInstance, functionNameString);
14293	#define VMA_FETCH_DEVICE_FUNC(memberName, functionPointerType, functionNameString) \
14294	if(m_VulkanFunctions.memberName == VMA_NULL) \
14295	m_VulkanFunctions.memberName = \
14296	(functionPointerType)m_VulkanFunctions.vkGetDeviceProcAddr(m_hDevice, functionNameString);
14297
14298	VMA_FETCH_INSTANCE_FUNC(vkGetPhysicalDeviceProperties, PFN_vkGetPhysicalDeviceProperties, "vkGetPhysicalDeviceProperties");
14299	VMA_FETCH_INSTANCE_FUNC(vkGetPhysicalDeviceMemoryProperties, PFN_vkGetPhysicalDeviceMemoryProperties, "vkGetPhysicalDeviceMemoryProperties");
14300	VMA_FETCH_DEVICE_FUNC(vkAllocateMemory, PFN_vkAllocateMemory, "vkAllocateMemory");
14301	VMA_FETCH_DEVICE_FUNC(vkFreeMemory, PFN_vkFreeMemory, "vkFreeMemory");
14302	VMA_FETCH_DEVICE_FUNC(vkMapMemory, PFN_vkMapMemory, "vkMapMemory");
14303	VMA_FETCH_DEVICE_FUNC(vkUnmapMemory, PFN_vkUnmapMemory, "vkUnmapMemory");
14304	VMA_FETCH_DEVICE_FUNC(vkFlushMappedMemoryRanges, PFN_vkFlushMappedMemoryRanges, "vkFlushMappedMemoryRanges");
14305	VMA_FETCH_DEVICE_FUNC(vkInvalidateMappedMemoryRanges, PFN_vkInvalidateMappedMemoryRanges, "vkInvalidateMappedMemoryRanges");
14306	VMA_FETCH_DEVICE_FUNC(vkBindBufferMemory, PFN_vkBindBufferMemory, "vkBindBufferMemory");
14307	VMA_FETCH_DEVICE_FUNC(vkBindImageMemory, PFN_vkBindImageMemory, "vkBindImageMemory");
14308	VMA_FETCH_DEVICE_FUNC(vkGetBufferMemoryRequirements, PFN_vkGetBufferMemoryRequirements, "vkGetBufferMemoryRequirements");
14309	VMA_FETCH_DEVICE_FUNC(vkGetImageMemoryRequirements, PFN_vkGetImageMemoryRequirements, "vkGetImageMemoryRequirements");
14310	VMA_FETCH_DEVICE_FUNC(vkCreateBuffer, PFN_vkCreateBuffer, "vkCreateBuffer");
14311	VMA_FETCH_DEVICE_FUNC(vkDestroyBuffer, PFN_vkDestroyBuffer, "vkDestroyBuffer");
14312	VMA_FETCH_DEVICE_FUNC(vkCreateImage, PFN_vkCreateImage, "vkCreateImage");
14313	VMA_FETCH_DEVICE_FUNC(vkDestroyImage, PFN_vkDestroyImage, "vkDestroyImage");
14314	VMA_FETCH_DEVICE_FUNC(vkCmdCopyBuffer, PFN_vkCmdCopyBuffer, "vkCmdCopyBuffer");
14315
14316	#if VMA_VULKAN_VERSION >= 1001000
14317	if(m_VulkanApiVersion >= VK_MAKE_VERSION(`1`, `1`, `0`))
14318	{
14319	VMA_FETCH_DEVICE_FUNC(vkGetBufferMemoryRequirements2KHR, PFN_vkGetBufferMemoryRequirements2, "vkGetBufferMemoryRequirements2");
14320	VMA_FETCH_DEVICE_FUNC(vkGetImageMemoryRequirements2KHR, PFN_vkGetImageMemoryRequirements2, "vkGetImageMemoryRequirements2");
14321	VMA_FETCH_DEVICE_FUNC(vkBindBufferMemory2KHR, PFN_vkBindBufferMemory2, "vkBindBufferMemory2");
14322	VMA_FETCH_DEVICE_FUNC(vkBindImageMemory2KHR, PFN_vkBindImageMemory2, "vkBindImageMemory2");
14323	VMA_FETCH_INSTANCE_FUNC(vkGetPhysicalDeviceMemoryProperties2KHR, PFN_vkGetPhysicalDeviceMemoryProperties2, "vkGetPhysicalDeviceMemoryProperties2");
14324	}
14325	#endif
14326
14327	#if VMA_DEDICATED_ALLOCATION
14328	if(m_UseKhrDedicatedAllocation)
14329	{
14330	VMA_FETCH_DEVICE_FUNC(vkGetBufferMemoryRequirements2KHR, PFN_vkGetBufferMemoryRequirements2KHR, "vkGetBufferMemoryRequirements2KHR");
14331	VMA_FETCH_DEVICE_FUNC(vkGetImageMemoryRequirements2KHR, PFN_vkGetImageMemoryRequirements2KHR, "vkGetImageMemoryRequirements2KHR");
14332	}
14333	#endif
14334
14335	#if VMA_BIND_MEMORY2
14336	if(m_UseKhrBindMemory2)
14337	{
14338	VMA_FETCH_DEVICE_FUNC(vkBindBufferMemory2KHR, PFN_vkBindBufferMemory2KHR, "vkBindBufferMemory2KHR");
14339	VMA_FETCH_DEVICE_FUNC(vkBindImageMemory2KHR, PFN_vkBindImageMemory2KHR, "vkBindImageMemory2KHR");
14340	}
14341	#endif // #if VMA_BIND_MEMORY2
14342
14343	#if VMA_MEMORY_BUDGET
14344	if(m_UseExtMemoryBudget)
14345	{
14346	VMA_FETCH_INSTANCE_FUNC(vkGetPhysicalDeviceMemoryProperties2KHR, PFN_vkGetPhysicalDeviceMemoryProperties2KHR, "vkGetPhysicalDeviceMemoryProperties2KHR");
14347	}
14348	#endif // #if VMA_MEMORY_BUDGET
14349
14350	#if VMA_VULKAN_VERSION >= 1003000
14351	if(m_VulkanApiVersion >= VK_MAKE_VERSION(`1`, `3`, `0`))
14352	{
14353	VMA_FETCH_DEVICE_FUNC(vkGetDeviceBufferMemoryRequirements, PFN_vkGetDeviceBufferMemoryRequirements, "vkGetDeviceBufferMemoryRequirements");
14354	VMA_FETCH_DEVICE_FUNC(vkGetDeviceImageMemoryRequirements, PFN_vkGetDeviceImageMemoryRequirements, "vkGetDeviceImageMemoryRequirements");
14355	}
14356	#endif
14357
14358	#undef VMA_FETCH_DEVICE_FUNC
14359	#undef VMA_FETCH_INSTANCE_FUNC
14360	}
14361
14362	#endif // VMA_DYNAMIC_VULKAN_FUNCTIONS == 1
14363
14364	void VmaAllocator_T::ValidateVulkanFunctions()
14365	{
14366	VMA_ASSERT(m_VulkanFunctions.vkGetPhysicalDeviceProperties != VMA_NULL);
14367	VMA_ASSERT(m_VulkanFunctions.vkGetPhysicalDeviceMemoryProperties != VMA_NULL);
14368	VMA_ASSERT(m_VulkanFunctions.vkAllocateMemory != VMA_NULL);
14369	VMA_ASSERT(m_VulkanFunctions.vkFreeMemory != VMA_NULL);
14370	VMA_ASSERT(m_VulkanFunctions.vkMapMemory != VMA_NULL);
14371	VMA_ASSERT(m_VulkanFunctions.vkUnmapMemory != VMA_NULL);
14372	VMA_ASSERT(m_VulkanFunctions.vkFlushMappedMemoryRanges != VMA_NULL);
14373	VMA_ASSERT(m_VulkanFunctions.vkInvalidateMappedMemoryRanges != VMA_NULL);
14374	VMA_ASSERT(m_VulkanFunctions.vkBindBufferMemory != VMA_NULL);
14375	VMA_ASSERT(m_VulkanFunctions.vkBindImageMemory != VMA_NULL);
14376	VMA_ASSERT(m_VulkanFunctions.vkGetBufferMemoryRequirements != VMA_NULL);
14377	VMA_ASSERT(m_VulkanFunctions.vkGetImageMemoryRequirements != VMA_NULL);
14378	VMA_ASSERT(m_VulkanFunctions.vkCreateBuffer != VMA_NULL);
14379	VMA_ASSERT(m_VulkanFunctions.vkDestroyBuffer != VMA_NULL);
14380	VMA_ASSERT(m_VulkanFunctions.vkCreateImage != VMA_NULL);
14381	VMA_ASSERT(m_VulkanFunctions.vkDestroyImage != VMA_NULL);
14382	VMA_ASSERT(m_VulkanFunctions.vkCmdCopyBuffer != VMA_NULL);
14383
14384	#if VMA_DEDICATED_ALLOCATION \|\| VMA_VULKAN_VERSION >= 1001000
14385	if(m_VulkanApiVersion >= VK_MAKE_VERSION(`1`, `1`, `0`) \|\| m_UseKhrDedicatedAllocation)
14386	{
14387	VMA_ASSERT(m_VulkanFunctions.vkGetBufferMemoryRequirements2KHR != VMA_NULL);
14388	VMA_ASSERT(m_VulkanFunctions.vkGetImageMemoryRequirements2KHR != VMA_NULL);
14389	}
14390	#endif
14391
14392	#if VMA_BIND_MEMORY2 \|\| VMA_VULKAN_VERSION >= 1001000
14393	if(m_VulkanApiVersion >= VK_MAKE_VERSION(`1`, `1`, `0`) \|\| m_UseKhrBindMemory2)
14394	{
14395	VMA_ASSERT(m_VulkanFunctions.vkBindBufferMemory2KHR != VMA_NULL);
14396	VMA_ASSERT(m_VulkanFunctions.vkBindImageMemory2KHR != VMA_NULL);
14397	}
14398	#endif
14399
14400	#if VMA_MEMORY_BUDGET \|\| VMA_VULKAN_VERSION >= 1001000
14401	if(m_UseExtMemoryBudget \|\| m_VulkanApiVersion >= VK_MAKE_VERSION(`1`, `1`, `0`))
14402	{
14403	VMA_ASSERT(m_VulkanFunctions.vkGetPhysicalDeviceMemoryProperties2KHR != VMA_NULL);
14404	}
14405	#endif
14406
14407	#if VMA_VULKAN_VERSION >= 1003000
14408	if(m_VulkanApiVersion >= VK_MAKE_VERSION(`1`, `3`, `0`))
14409	{
14410	VMA_ASSERT(m_VulkanFunctions.vkGetDeviceBufferMemoryRequirements != VMA_NULL);
14411	VMA_ASSERT(m_VulkanFunctions.vkGetDeviceImageMemoryRequirements != VMA_NULL);
14412	}
14413	#endif
14414	}
14415
14416	VkDeviceSize VmaAllocator_T::CalcPreferredBlockSize(uint32_t memTypeIndex)
14417	{
14418	const uint32_t heapIndex = MemoryTypeIndexToHeapIndex(memTypeIndex);
14419	const VkDeviceSize heapSize = m_MemProps.memoryHeaps[heapIndex].size;
14420	const bool isSmallHeap = heapSize <= VMA_SMALL_HEAP_MAX_SIZE;
14421	return VmaAlignUp(isSmallHeap ? (heapSize / `8`) : m_PreferredLargeHeapBlockSize, (VkDeviceSize)`32`);
14422	}
14423
14424	VkResult VmaAllocator_T::AllocateMemoryOfType(
14425	VmaPool pool,
14426	VkDeviceSize size,
14427	VkDeviceSize alignment,
14428	bool dedicatedPreferred,
14429	VkBuffer dedicatedBuffer,
14430	VkImage dedicatedImage,
14431	VkFlags dedicatedBufferImageUsage,
14432	const VmaAllocationCreateInfo& createInfo,
14433	uint32_t memTypeIndex,
14434	VmaSuballocationType suballocType,
14435	VmaDedicatedAllocationList& dedicatedAllocations,
14436	VmaBlockVector& blockVector,
14437	size_t allocationCount,
14438	VmaAllocation* pAllocations)
14439	{
14440	VMA_ASSERT(pAllocations != VMA_NULL);
14441	VMA_DEBUG_LOG(" AllocateMemory: MemoryTypeIndex=%u, AllocationCount=%zu, Size=%llu", memTypeIndex, allocationCount, size);
14442
14443	VmaAllocationCreateInfo finalCreateInfo = createInfo;
14444	VkResult res = CalcMemTypeParams(
14445	finalCreateInfo,
14446	memTypeIndex,
14447	size,
14448	allocationCount);
14449	if(res != VK_SUCCESS)
14450	return res;
14451
14452	if((finalCreateInfo.flags & VMA_ALLOCATION_CREATE_DEDICATED_MEMORY_BIT) != `0`)
14453	{
14454	return AllocateDedicatedMemory(
14455	pool,
14456	size,
14457	suballocType,
14458	dedicatedAllocations,
14459	memTypeIndex,
14460	(finalCreateInfo.flags & VMA_ALLOCATION_CREATE_MAPPED_BIT) != `0`,
14461	(finalCreateInfo.flags & VMA_ALLOCATION_CREATE_USER_DATA_COPY_STRING_BIT) != `0`,
14462	(finalCreateInfo.flags &
14463	(VMA_ALLOCATION_CREATE_HOST_ACCESS_SEQUENTIAL_WRITE_BIT \| VMA_ALLOCATION_CREATE_HOST_ACCESS_RANDOM_BIT)) != `0`,
14464	(finalCreateInfo.flags & VMA_ALLOCATION_CREATE_CAN_ALIAS_BIT) != `0`,
14465	finalCreateInfo.pUserData,
14466	finalCreateInfo.priority,
14467	dedicatedBuffer,
14468	dedicatedImage,
14469	dedicatedBufferImageUsage,
14470	allocationCount,
14471	pAllocations,
14472	blockVector.GetAllocationNextPtr());
14473	}
14474	else
14475	{
14476	const bool canAllocateDedicated =
14477	(finalCreateInfo.flags & VMA_ALLOCATION_CREATE_NEVER_ALLOCATE_BIT) == `0` &&
14478	(pool == VK_NULL_HANDLE \|\| !blockVector.HasExplicitBlockSize());
14479
14480	if(canAllocateDedicated)
14481	{
14482	// Heuristics: Allocate dedicated memory if requested size if greater than half of preferred block size.
14483	if(size > blockVector.GetPreferredBlockSize() / `2`)
14484	{
14485	dedicatedPreferred = true;
14486	}
14487	// Protection against creating each allocation as dedicated when we reach or exceed heap size/budget,
14488	// which can quickly deplete maxMemoryAllocationCount: Don't prefer dedicated allocations when above
14489	// 3/4 of the maximum allocation count.
14490	if(m_DeviceMemoryCount.load() > m_PhysicalDeviceProperties.limits.maxMemoryAllocationCount * `3` / `4`)
14491	{
14492	dedicatedPreferred = false;
14493	}
14494
14495	if(dedicatedPreferred)
14496	{
14497	res = AllocateDedicatedMemory(
14498	pool,
14499	size,
14500	suballocType,
14501	dedicatedAllocations,
14502	memTypeIndex,
14503	(finalCreateInfo.flags & VMA_ALLOCATION_CREATE_MAPPED_BIT) != `0`,
14504	(finalCreateInfo.flags & VMA_ALLOCATION_CREATE_USER_DATA_COPY_STRING_BIT) != `0`,
14505	(finalCreateInfo.flags &
14506	(VMA_ALLOCATION_CREATE_HOST_ACCESS_SEQUENTIAL_WRITE_BIT \| VMA_ALLOCATION_CREATE_HOST_ACCESS_RANDOM_BIT)) != `0`,
14507	(finalCreateInfo.flags & VMA_ALLOCATION_CREATE_CAN_ALIAS_BIT) != `0`,
14508	finalCreateInfo.pUserData,
14509	finalCreateInfo.priority,
14510	dedicatedBuffer,
14511	dedicatedImage,
14512	dedicatedBufferImageUsage,
14513	allocationCount,
14514	pAllocations,
14515	blockVector.GetAllocationNextPtr());
14516	if(res == VK_SUCCESS)
14517	{
14518	// Succeeded: AllocateDedicatedMemory function already filld pMemory, nothing more to do here.
14519	VMA_DEBUG_LOG(" Allocated as DedicatedMemory");
14520	return VK_SUCCESS;
14521	}
14522	}
14523	}
14524
14525	res = blockVector.Allocate(
14526	size,
14527	alignment,
14528	finalCreateInfo,
14529	suballocType,
14530	allocationCount,
14531	pAllocations);
14532	if(res == VK_SUCCESS)
14533	return VK_SUCCESS;
14534
14535	// Try dedicated memory.
14536	if(canAllocateDedicated && !dedicatedPreferred)
14537	{
14538	res = AllocateDedicatedMemory(
14539	pool,
14540	size,
14541	suballocType,
14542	dedicatedAllocations,
14543	memTypeIndex,
14544	(finalCreateInfo.flags & VMA_ALLOCATION_CREATE_MAPPED_BIT) != `0`,
14545	(finalCreateInfo.flags & VMA_ALLOCATION_CREATE_USER_DATA_COPY_STRING_BIT) != `0`,
14546	(finalCreateInfo.flags &
14547	(VMA_ALLOCATION_CREATE_HOST_ACCESS_SEQUENTIAL_WRITE_BIT \| VMA_ALLOCATION_CREATE_HOST_ACCESS_RANDOM_BIT)) != `0`,
14548	(finalCreateInfo.flags & VMA_ALLOCATION_CREATE_CAN_ALIAS_BIT) != `0`,
14549	finalCreateInfo.pUserData,
14550	finalCreateInfo.priority,
14551	dedicatedBuffer,
14552	dedicatedImage,
14553	dedicatedBufferImageUsage,
14554	allocationCount,
14555	pAllocations,
14556	blockVector.GetAllocationNextPtr());
14557	if(res == VK_SUCCESS)
14558	{
14559	// Succeeded: AllocateDedicatedMemory function already filld pMemory, nothing more to do here.
14560	VMA_DEBUG_LOG(" Allocated as DedicatedMemory");
14561	return VK_SUCCESS;
14562	}
14563	}
14564	// Everything failed: Return error code.
14565	VMA_DEBUG_LOG(" vkAllocateMemory FAILED");
14566	return res;
14567	}
14568	}
14569
14570	VkResult VmaAllocator_T::AllocateDedicatedMemory(
14571	VmaPool pool,
14572	VkDeviceSize size,
14573	VmaSuballocationType suballocType,
14574	VmaDedicatedAllocationList& dedicatedAllocations,
14575	uint32_t memTypeIndex,
14576	bool map,
14577	bool isUserDataString,
14578	bool isMappingAllowed,
14579	bool canAliasMemory,
14580	void* pUserData,
14581	float priority,
14582	VkBuffer dedicatedBuffer,
14583	VkImage dedicatedImage,
14584	VkFlags dedicatedBufferImageUsage,
14585	size_t allocationCount,
14586	VmaAllocation* pAllocations,
14587	const void* pNextChain)
14588	{
14589	VMA_ASSERT(allocationCount > `0` && pAllocations);
14590
14591	VkMemoryAllocateInfo allocInfo = { VK_STRUCTURE_TYPE_MEMORY_ALLOCATE_INFO };
14592	allocInfo.memoryTypeIndex = memTypeIndex;
14593	allocInfo.allocationSize = size;
14594	allocInfo.pNext = pNextChain;
14595
14596	#if VMA_DEDICATED_ALLOCATION \|\| VMA_VULKAN_VERSION >= 1001000
14597	VkMemoryDedicatedAllocateInfoKHR dedicatedAllocInfo = { VK_STRUCTURE_TYPE_MEMORY_DEDICATED_ALLOCATE_INFO_KHR };
14598	if(!canAliasMemory)
14599	{
14600	if(m_UseKhrDedicatedAllocation \|\| m_VulkanApiVersion >= VK_MAKE_VERSION(`1`, `1`, `0`))
14601	{
14602	if(dedicatedBuffer != VK_NULL_HANDLE)
14603	{
14604	VMA_ASSERT(dedicatedImage == VK_NULL_HANDLE);
14605	dedicatedAllocInfo.buffer = dedicatedBuffer;
14606	VmaPnextChainPushFront(&allocInfo, &dedicatedAllocInfo);
14607	}
14608	else if(dedicatedImage != VK_NULL_HANDLE)
14609	{
14610	dedicatedAllocInfo.image = dedicatedImage;
14611	VmaPnextChainPushFront(&allocInfo, &dedicatedAllocInfo);
14612	}
14613	}
14614	}
14615	#endif // #if VMA_DEDICATED_ALLOCATION \|\| VMA_VULKAN_VERSION >= 1001000
14616
14617	#if VMA_BUFFER_DEVICE_ADDRESS
14618	VkMemoryAllocateFlagsInfoKHR allocFlagsInfo = { VK_STRUCTURE_TYPE_MEMORY_ALLOCATE_FLAGS_INFO_KHR };
14619	if(m_UseKhrBufferDeviceAddress)
14620	{
14621	bool canContainBufferWithDeviceAddress = true;
14622	if(dedicatedBuffer != VK_NULL_HANDLE)
14623	{
14624	canContainBufferWithDeviceAddress = dedicatedBufferImageUsage == UINT32_MAX \|\| // Usage flags unknown
14625	(dedicatedBufferImageUsage & VK_BUFFER_USAGE_SHADER_DEVICE_ADDRESS_BIT_EXT) != `0`;
14626	}
14627	else if(dedicatedImage != VK_NULL_HANDLE)
14628	{
14629	canContainBufferWithDeviceAddress = false;
14630	}
14631	if(canContainBufferWithDeviceAddress)
14632	{
14633	allocFlagsInfo.flags = VK_MEMORY_ALLOCATE_DEVICE_ADDRESS_BIT_KHR;
14634	VmaPnextChainPushFront(&allocInfo, &allocFlagsInfo);
14635	}
14636	}
14637	#endif // #if VMA_BUFFER_DEVICE_ADDRESS
14638
14639	#if VMA_MEMORY_PRIORITY
14640	VkMemoryPriorityAllocateInfoEXT priorityInfo = { VK_STRUCTURE_TYPE_MEMORY_PRIORITY_ALLOCATE_INFO_EXT };
14641	if(m_UseExtMemoryPriority)
14642	{
14643	VMA_ASSERT(priority >= `0.f` && priority <= `1.f`);
14644	priorityInfo.priority = priority;
14645	VmaPnextChainPushFront(&allocInfo, &priorityInfo);
14646	}
14647	#endif // #if VMA_MEMORY_PRIORITY
14648
14649	#if VMA_EXTERNAL_MEMORY
14650	// Attach VkExportMemoryAllocateInfoKHR if necessary.
14651	VkExportMemoryAllocateInfoKHR exportMemoryAllocInfo = { VK_STRUCTURE_TYPE_EXPORT_MEMORY_ALLOCATE_INFO_KHR };
14652	exportMemoryAllocInfo.handleTypes = GetExternalMemoryHandleTypeFlags(memTypeIndex);
14653	if(exportMemoryAllocInfo.handleTypes != `0`)
14654	{
14655	VmaPnextChainPushFront(&allocInfo, &exportMemoryAllocInfo);
14656	}
14657	#endif // #if VMA_EXTERNAL_MEMORY
14658
14659	size_t allocIndex;
14660	VkResult res = VK_SUCCESS;
14661	for(allocIndex = `0`; allocIndex < allocationCount; ++allocIndex)
14662	{
14663	res = AllocateDedicatedMemoryPage(
14664	pool,
14665	size,
14666	suballocType,
14667	memTypeIndex,
14668	allocInfo,
14669	map,
14670	isUserDataString,
14671	isMappingAllowed,
14672	pUserData,
14673	pAllocations + allocIndex);
14674	if(res != VK_SUCCESS)
14675	{
14676	break;
14677	}
14678	}
14679
14680	if(res == VK_SUCCESS)
14681	{
14682	for (allocIndex = `0`; allocIndex < allocationCount; ++allocIndex)
14683	{
14684	dedicatedAllocations.Register(pAllocations[allocIndex]);
14685	}
14686	VMA_DEBUG_LOG(" Allocated DedicatedMemory Count=%zu, MemoryTypeIndex=#%u", allocationCount, memTypeIndex);
14687	}
14688	else
14689	{
14690	// Free all already created allocations.
14691	while(allocIndex--)
14692	{
14693	VmaAllocation currAlloc = pAllocations[allocIndex];
14694	VkDeviceMemory hMemory = currAlloc->GetMemory();
14695
14696	/*
14697	There is no need to call this, because Vulkan spec allows to skip vkUnmapMemory
14698	before vkFreeMemory.
14699
14700	if(currAlloc->GetMappedData() != VMA_NULL)
14701	{
14702	(m_VulkanFunctions.vkUnmapMemory)(m_hDevice, hMemory);*
14703	}
14704	*/
14705
14706	FreeVulkanMemory(memTypeIndex, currAlloc->GetSize(), hMemory);
14707	m_Budget.RemoveAllocation(MemoryTypeIndexToHeapIndex(memTypeIndex), currAlloc->GetSize());
14708	m_AllocationObjectAllocator.Free(currAlloc);
14709	}
14710
14711	memset(pAllocations, `0`, sizeof(VmaAllocation) * allocationCount);
14712	}
14713
14714	return res;
14715	}
14716
14717	VkResult VmaAllocator_T::AllocateDedicatedMemoryPage(
14718	VmaPool pool,
14719	VkDeviceSize size,
14720	VmaSuballocationType suballocType,
14721	uint32_t memTypeIndex,
14722	const VkMemoryAllocateInfo& allocInfo,
14723	bool map,
14724	bool isUserDataString,
14725	bool isMappingAllowed,
14726	void* pUserData,
14727	VmaAllocation* pAllocation)
14728	{
14729	VkDeviceMemory hMemory = VK_NULL_HANDLE;
14730	VkResult res = AllocateVulkanMemory(&allocInfo, &hMemory);
14731	if(res < `0`)
14732	{
14733	VMA_DEBUG_LOG(" vkAllocateMemory FAILED");
14734	return res;
14735	}
14736
14737	void* pMappedData = VMA_NULL;
14738	if(map)
14739	{
14740	res = (*m_VulkanFunctions.vkMapMemory)(
14741	m_hDevice,
14742	hMemory,
14743	`0`,
14744	VK_WHOLE_SIZE,
14745	`0`,
14746	&pMappedData);
14747	if(res < `0`)
14748	{
14749	VMA_DEBUG_LOG(" vkMapMemory FAILED");
14750	FreeVulkanMemory(memTypeIndex, size, hMemory);
14751	return res;
14752	}
14753	}
14754
14755	*pAllocation = m_AllocationObjectAllocator.Allocate(isMappingAllowed);
14756	(*pAllocation)->InitDedicatedAllocation(pool, memTypeIndex, hMemory, suballocType, pMappedData, size);
14757	if (isUserDataString)
14758	(pAllocation)->SetName(this, (const* char*)pUserData);
14759	else
14760	(pAllocation)->SetUserData(this*, pUserData);
14761	m_Budget.AddAllocation(MemoryTypeIndexToHeapIndex(memTypeIndex), size);
14762	if(VMA_DEBUG_INITIALIZE_ALLOCATIONS)
14763	{
14764	FillAllocation(*pAllocation, VMA_ALLOCATION_FILL_PATTERN_CREATED);
14765	}
14766
14767	return VK_SUCCESS;
14768	}
14769
14770	void VmaAllocator_T::GetBufferMemoryRequirements(
14771	VkBuffer hBuffer,
14772	VkMemoryRequirements& memReq,
14773	bool& requiresDedicatedAllocation,
14774	bool& prefersDedicatedAllocation) const
14775	{
14776	#if VMA_DEDICATED_ALLOCATION \|\| VMA_VULKAN_VERSION >= 1001000
14777	if(m_UseKhrDedicatedAllocation \|\| m_VulkanApiVersion >= VK_MAKE_VERSION(`1`, `1`, `0`))
14778	{
14779	VkBufferMemoryRequirementsInfo2KHR memReqInfo = { VK_STRUCTURE_TYPE_BUFFER_MEMORY_REQUIREMENTS_INFO_2_KHR };
14780	memReqInfo.buffer = hBuffer;
14781
14782	VkMemoryDedicatedRequirementsKHR memDedicatedReq = { VK_STRUCTURE_TYPE_MEMORY_DEDICATED_REQUIREMENTS_KHR };
14783
14784	VkMemoryRequirements2KHR memReq2 = { VK_STRUCTURE_TYPE_MEMORY_REQUIREMENTS_2_KHR };
14785	VmaPnextChainPushFront(&memReq2, &memDedicatedReq);
14786
14787	(*m_VulkanFunctions.vkGetBufferMemoryRequirements2KHR)(m_hDevice, &memReqInfo, &memReq2);
14788
14789	memReq = memReq2.memoryRequirements;
14790	requiresDedicatedAllocation = (memDedicatedReq.requiresDedicatedAllocation != VK_FALSE);
14791	prefersDedicatedAllocation = (memDedicatedReq.prefersDedicatedAllocation != VK_FALSE);
14792	}
14793	else
14794	#endif // #if VMA_DEDICATED_ALLOCATION \|\| VMA_VULKAN_VERSION >= 1001000
14795	{
14796	(*m_VulkanFunctions.vkGetBufferMemoryRequirements)(m_hDevice, hBuffer, &memReq);
14797	requiresDedicatedAllocation = false;
14798	prefersDedicatedAllocation = false;
14799	}
14800	}
14801
14802	void VmaAllocator_T::GetImageMemoryRequirements(
14803	VkImage hImage,
14804	VkMemoryRequirements& memReq,
14805	bool& requiresDedicatedAllocation,
14806	bool& prefersDedicatedAllocation) const
14807	{
14808	#if VMA_DEDICATED_ALLOCATION \|\| VMA_VULKAN_VERSION >= 1001000
14809	if(m_UseKhrDedicatedAllocation \|\| m_VulkanApiVersion >= VK_MAKE_VERSION(`1`, `1`, `0`))
14810	{
14811	VkImageMemoryRequirementsInfo2KHR memReqInfo = { VK_STRUCTURE_TYPE_IMAGE_MEMORY_REQUIREMENTS_INFO_2_KHR };
14812	memReqInfo.image = hImage;
14813
14814	VkMemoryDedicatedRequirementsKHR memDedicatedReq = { VK_STRUCTURE_TYPE_MEMORY_DEDICATED_REQUIREMENTS_KHR };
14815
14816	VkMemoryRequirements2KHR memReq2 = { VK_STRUCTURE_TYPE_MEMORY_REQUIREMENTS_2_KHR };
14817	VmaPnextChainPushFront(&memReq2, &memDedicatedReq);
14818
14819	(*m_VulkanFunctions.vkGetImageMemoryRequirements2KHR)(m_hDevice, &memReqInfo, &memReq2);
14820
14821	memReq = memReq2.memoryRequirements;
14822	requiresDedicatedAllocation = (memDedicatedReq.requiresDedicatedAllocation != VK_FALSE);
14823	prefersDedicatedAllocation = (memDedicatedReq.prefersDedicatedAllocation != VK_FALSE);
14824	}
14825	else
14826	#endif // #if VMA_DEDICATED_ALLOCATION \|\| VMA_VULKAN_VERSION >= 1001000
14827	{
14828	(*m_VulkanFunctions.vkGetImageMemoryRequirements)(m_hDevice, hImage, &memReq);
14829	requiresDedicatedAllocation = false;
14830	prefersDedicatedAllocation = false;
14831	}
14832	}
14833
14834	VkResult VmaAllocator_T::FindMemoryTypeIndex(
14835	uint32_t memoryTypeBits,
14836	const VmaAllocationCreateInfo* pAllocationCreateInfo,
14837	VkFlags bufImgUsage,
14838	uint32_t* pMemoryTypeIndex) const
14839	{
14840	memoryTypeBits &= GetGlobalMemoryTypeBits();
14841
14842	if(pAllocationCreateInfo->memoryTypeBits != `0`)
14843	{
14844	memoryTypeBits &= pAllocationCreateInfo->memoryTypeBits;
14845	}
14846
14847	VkMemoryPropertyFlags requiredFlags = `0`, preferredFlags = `0`, notPreferredFlags = `0`;
14848	if(!FindMemoryPreferences(
14849	IsIntegratedGpu(),
14850	*pAllocationCreateInfo,
14851	bufImgUsage,
14852	requiredFlags, preferredFlags, notPreferredFlags))
14853	{
14854	return VK_ERROR_FEATURE_NOT_PRESENT;
14855	}
14856
14857	*pMemoryTypeIndex = UINT32_MAX;
14858	uint32_t minCost = UINT32_MAX;
14859	for(uint32_t memTypeIndex = `0`, memTypeBit = `1`;
14860	memTypeIndex < GetMemoryTypeCount();
14861	++memTypeIndex, memTypeBit <<= `1`)
14862	{
14863	// This memory type is acceptable according to memoryTypeBits bitmask.
14864	if((memTypeBit & memoryTypeBits) != `0`)
14865	{
14866	const VkMemoryPropertyFlags currFlags =
14867	m_MemProps.memoryTypes[memTypeIndex].propertyFlags;
14868	// This memory type contains requiredFlags.
14869	if((requiredFlags & ~currFlags) == `0`)
14870	{
14871	// Calculate cost as number of bits from preferredFlags not present in this memory type.
14872	uint32_t currCost = VMA_COUNT_BITS_SET(preferredFlags & ~currFlags) +
14873	VMA_COUNT_BITS_SET(currFlags & notPreferredFlags);
14874	// Remember memory type with lowest cost.
14875	if(currCost < minCost)
14876	{
14877	*pMemoryTypeIndex = memTypeIndex;
14878	if(currCost == `0`)
14879	{
14880	return VK_SUCCESS;
14881	}
14882	minCost = currCost;
14883	}
14884	}
14885	}
14886	}
14887	return (*pMemoryTypeIndex != UINT32_MAX) ? VK_SUCCESS : VK_ERROR_FEATURE_NOT_PRESENT;
14888	}
14889
14890	VkResult VmaAllocator_T::CalcMemTypeParams(
14891	VmaAllocationCreateInfo& inoutCreateInfo,
14892	uint32_t memTypeIndex,
14893	VkDeviceSize size,
14894	size_t allocationCount)
14895	{
14896	// If memory type is not HOST_VISIBLE, disable MAPPED.
14897	if((inoutCreateInfo.flags & VMA_ALLOCATION_CREATE_MAPPED_BIT) != `0` &&
14898	(m_MemProps.memoryTypes[memTypeIndex].propertyFlags & VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT) == `0`)
14899	{
14900	inoutCreateInfo.flags &= ~VMA_ALLOCATION_CREATE_MAPPED_BIT;
14901	}
14902
14903	if((inoutCreateInfo.flags & VMA_ALLOCATION_CREATE_DEDICATED_MEMORY_BIT) != `0` &&
14904	(inoutCreateInfo.flags & VMA_ALLOCATION_CREATE_WITHIN_BUDGET_BIT) != `0`)
14905	{
14906	const uint32_t heapIndex = MemoryTypeIndexToHeapIndex(memTypeIndex);
14907	VmaBudget heapBudget = {};
14908	GetHeapBudgets(&heapBudget, heapIndex, `1`);
14909	if(heapBudget.usage + size * allocationCount > heapBudget.budget)
14910	{
14911	return VK_ERROR_OUT_OF_DEVICE_MEMORY;
14912	}
14913	}
14914	return VK_SUCCESS;
14915	}
14916
14917	VkResult VmaAllocator_T::CalcAllocationParams(
14918	VmaAllocationCreateInfo& inoutCreateInfo,
14919	bool dedicatedRequired,
14920	bool dedicatedPreferred)
14921	{
14922	VMA_ASSERT((inoutCreateInfo.flags &
14923	(VMA_ALLOCATION_CREATE_HOST_ACCESS_SEQUENTIAL_WRITE_BIT \| VMA_ALLOCATION_CREATE_HOST_ACCESS_RANDOM_BIT)) !=
14924	(VMA_ALLOCATION_CREATE_HOST_ACCESS_SEQUENTIAL_WRITE_BIT \| VMA_ALLOCATION_CREATE_HOST_ACCESS_RANDOM_BIT) &&
14925	"Specifying both flags VMA_ALLOCATION_CREATE_HOST_ACCESS_SEQUENTIAL_WRITE_BIT and VMA_ALLOCATION_CREATE_HOST_ACCESS_RANDOM_BIT is incorrect.");
14926	VMA_ASSERT((((inoutCreateInfo.flags & VMA_ALLOCATION_CREATE_HOST_ACCESS_ALLOW_TRANSFER_INSTEAD_BIT) == `0` \|\|
14927	(inoutCreateInfo.flags & (VMA_ALLOCATION_CREATE_HOST_ACCESS_SEQUENTIAL_WRITE_BIT \| VMA_ALLOCATION_CREATE_HOST_ACCESS_RANDOM_BIT)) != `0`)) &&
14928	"Specifying VMA_ALLOCATION_CREATE_HOST_ACCESS_ALLOW_TRANSFER_INSTEAD_BIT requires also VMA_ALLOCATION_CREATE_HOST_ACCESS_SEQUENTIAL_WRITE_BIT or VMA_ALLOCATION_CREATE_HOST_ACCESS_RANDOM_BIT.");
14929	if(inoutCreateInfo.usage == VMA_MEMORY_USAGE_AUTO \|\| inoutCreateInfo.usage == VMA_MEMORY_USAGE_AUTO_PREFER_DEVICE \|\| inoutCreateInfo.usage == VMA_MEMORY_USAGE_AUTO_PREFER_HOST)
14930	{
14931	if((inoutCreateInfo.flags & VMA_ALLOCATION_CREATE_MAPPED_BIT) != `0`)
14932	{
14933	VMA_ASSERT((inoutCreateInfo.flags & (VMA_ALLOCATION_CREATE_HOST_ACCESS_SEQUENTIAL_WRITE_BIT \| VMA_ALLOCATION_CREATE_HOST_ACCESS_RANDOM_BIT)) != `0` &&
14934	"When using VMA_ALLOCATION_CREATE_MAPPED_BIT and usage = VMA_MEMORY_USAGE_AUTO*, you must also specify VMA_ALLOCATION_CREATE_HOST_ACCESS_SEQUENTIAL_WRITE_BIT or VMA_ALLOCATION_CREATE_HOST_ACCESS_RANDOM_BIT.");
14935	}
14936	}
14937
14938	// If memory is lazily allocated, it should be always dedicated.
14939	if(dedicatedRequired \|\|
14940	inoutCreateInfo.usage == VMA_MEMORY_USAGE_GPU_LAZILY_ALLOCATED)
14941	{
14942	inoutCreateInfo.flags \|= VMA_ALLOCATION_CREATE_DEDICATED_MEMORY_BIT;
14943	}
14944
14945	if(inoutCreateInfo.pool != VK_NULL_HANDLE)
14946	{
14947	if(inoutCreateInfo.pool->m_BlockVector.HasExplicitBlockSize() &&
14948	(inoutCreateInfo.flags & VMA_ALLOCATION_CREATE_DEDICATED_MEMORY_BIT) != `0`)
14949	{
14950	VMA_ASSERT(`0` && "Specifying VMA_ALLOCATION_CREATE_DEDICATED_MEMORY_BIT while current custom pool doesn't support dedicated allocations.");
14951	return VK_ERROR_FEATURE_NOT_PRESENT;
14952	}
14953	inoutCreateInfo.priority = inoutCreateInfo.pool->m_BlockVector.GetPriority();
14954	}
14955
14956	if((inoutCreateInfo.flags & VMA_ALLOCATION_CREATE_DEDICATED_MEMORY_BIT) != `0` &&
14957	(inoutCreateInfo.flags & VMA_ALLOCATION_CREATE_NEVER_ALLOCATE_BIT) != `0`)
14958	{
14959	VMA_ASSERT(`0` && "Specifying VMA_ALLOCATION_CREATE_DEDICATED_MEMORY_BIT together with VMA_ALLOCATION_CREATE_NEVER_ALLOCATE_BIT makes no sense.");
14960	return VK_ERROR_FEATURE_NOT_PRESENT;
14961	}
14962
14963	if(VMA_DEBUG_ALWAYS_DEDICATED_MEMORY &&
14964	(inoutCreateInfo.flags & VMA_ALLOCATION_CREATE_NEVER_ALLOCATE_BIT) != `0`)
14965	{
14966	inoutCreateInfo.flags \|= VMA_ALLOCATION_CREATE_DEDICATED_MEMORY_BIT;
14967	}
14968
14969	// Non-auto USAGE values imply HOST_ACCESS flags.
14970	// And so does VMA_MEMORY_USAGE_UNKNOWN because it is used with custom pools.
14971	// Which specific flag is used doesn't matter. They change things only when used with VMA_MEMORY_USAGE_AUTO.*
14972	// Otherwise they just protect from assert on mapping.
14973	if(inoutCreateInfo.usage != VMA_MEMORY_USAGE_AUTO &&
14974	inoutCreateInfo.usage != VMA_MEMORY_USAGE_AUTO_PREFER_DEVICE &&
14975	inoutCreateInfo.usage != VMA_MEMORY_USAGE_AUTO_PREFER_HOST)
14976	{
14977	if((inoutCreateInfo.flags & (VMA_ALLOCATION_CREATE_HOST_ACCESS_SEQUENTIAL_WRITE_BIT \| VMA_ALLOCATION_CREATE_HOST_ACCESS_RANDOM_BIT)) == `0`)
14978	{
14979	inoutCreateInfo.flags \|= VMA_ALLOCATION_CREATE_HOST_ACCESS_RANDOM_BIT;
14980	}
14981	}
14982
14983	return VK_SUCCESS;
14984	}
14985
14986	VkResult VmaAllocator_T::AllocateMemory(
14987	const VkMemoryRequirements& vkMemReq,
14988	bool requiresDedicatedAllocation,
14989	bool prefersDedicatedAllocation,
14990	VkBuffer dedicatedBuffer,
14991	VkImage dedicatedImage,
14992	VkFlags dedicatedBufferImageUsage,
14993	const VmaAllocationCreateInfo& createInfo,
14994	VmaSuballocationType suballocType,
14995	size_t allocationCount,
14996	VmaAllocation* pAllocations)
14997	{
14998	memset(pAllocations, `0`, sizeof(VmaAllocation) * allocationCount);
14999
15000	VMA_ASSERT(VmaIsPow2(vkMemReq.alignment));
15001
15002	if(vkMemReq.size == `0`)
15003	{
15004	return VK_ERROR_INITIALIZATION_FAILED;
15005	}
15006
15007	VmaAllocationCreateInfo createInfoFinal = createInfo;
15008	VkResult res = CalcAllocationParams(createInfoFinal, requiresDedicatedAllocation, prefersDedicatedAllocation);
15009	if(res != VK_SUCCESS)
15010	return res;
15011
15012	if(createInfoFinal.pool != VK_NULL_HANDLE)
15013	{
15014	VmaBlockVector& blockVector = createInfoFinal.pool->m_BlockVector;
15015	return AllocateMemoryOfType(
15016	createInfoFinal.pool,
15017	vkMemReq.size,
15018	vkMemReq.alignment,
15019	prefersDedicatedAllocation,
15020	dedicatedBuffer,
15021	dedicatedImage,
15022	dedicatedBufferImageUsage,
15023	createInfoFinal,
15024	blockVector.GetMemoryTypeIndex(),
15025	suballocType,
15026	createInfoFinal.pool->m_DedicatedAllocations,
15027	blockVector,
15028	allocationCount,
15029	pAllocations);
15030	}
15031	else
15032	{
15033	// Bit mask of memory Vulkan types acceptable for this allocation.
15034	uint32_t memoryTypeBits = vkMemReq.memoryTypeBits;
15035	uint32_t memTypeIndex = UINT32_MAX;
15036	res = FindMemoryTypeIndex(memoryTypeBits, &createInfoFinal, dedicatedBufferImageUsage, &memTypeIndex);
15037	// Can't find any single memory type matching requirements. res is VK_ERROR_FEATURE_NOT_PRESENT.
15038	if(res != VK_SUCCESS)
15039	return res;
15040	do
15041	{
15042	VmaBlockVector* blockVector = m_pBlockVectors[memTypeIndex];
15043	VMA_ASSERT(blockVector && "Trying to use unsupported memory type!");
15044	res = AllocateMemoryOfType(
15045	VK_NULL_HANDLE,
15046	vkMemReq.size,
15047	vkMemReq.alignment,
15048	requiresDedicatedAllocation \|\| prefersDedicatedAllocation,
15049	dedicatedBuffer,
15050	dedicatedImage,
15051	dedicatedBufferImageUsage,
15052	createInfoFinal,
15053	memTypeIndex,
15054	suballocType,
15055	m_DedicatedAllocations[memTypeIndex],
15056	*blockVector,
15057	allocationCount,
15058	pAllocations);
15059	// Allocation succeeded
15060	if(res == VK_SUCCESS)
15061	return VK_SUCCESS;
15062
15063	// Remove old memTypeIndex from list of possibilities.
15064	memoryTypeBits &= ~(`1u` << memTypeIndex);
15065	// Find alternative memTypeIndex.
15066	res = FindMemoryTypeIndex(memoryTypeBits, &createInfoFinal, dedicatedBufferImageUsage, &memTypeIndex);
15067	} while(res == VK_SUCCESS);
15068
15069	// No other matching memory type index could be found.
15070	// Not returning res, which is VK_ERROR_FEATURE_NOT_PRESENT, because we already failed to allocate once.
15071	return VK_ERROR_OUT_OF_DEVICE_MEMORY;
15072	}
15073	}
15074
15075	void VmaAllocator_T::FreeMemory(
15076	size_t allocationCount,
15077	const VmaAllocation* pAllocations)
15078	{
15079	VMA_ASSERT(pAllocations);
15080
15081	for(size_t allocIndex = allocationCount; allocIndex--; )
15082	{
15083	VmaAllocation allocation = pAllocations[allocIndex];
15084
15085	if(allocation != VK_NULL_HANDLE)
15086	{
15087	if(VMA_DEBUG_INITIALIZE_ALLOCATIONS)
15088	{
15089	FillAllocation(allocation, VMA_ALLOCATION_FILL_PATTERN_DESTROYED);
15090	}
15091
15092	allocation->FreeName(this);
15093
15094	switch(allocation->GetType())
15095	{
15096	case VmaAllocation_T::ALLOCATION_TYPE_BLOCK:
15097	{
15098	VmaBlockVector* pBlockVector = VMA_NULL;
15099	VmaPool hPool = allocation->GetParentPool();
15100	if(hPool != VK_NULL_HANDLE)
15101	{
15102	pBlockVector = &hPool->m_BlockVector;
15103	}
15104	else
15105	{
15106	const uint32_t memTypeIndex = allocation->GetMemoryTypeIndex();
15107	pBlockVector = m_pBlockVectors[memTypeIndex];
15108	VMA_ASSERT(pBlockVector && "Trying to free memory of unsupported type!");
15109	}
15110	pBlockVector->Free(allocation);
15111	}
15112	break;
15113	case VmaAllocation_T::ALLOCATION_TYPE_DEDICATED:
15114	FreeDedicatedMemory(allocation);
15115	break;
15116	default:
15117	VMA_ASSERT(`0`);
15118	}
15119	}
15120	}
15121	}
15122
15123	void VmaAllocator_T::CalculateStatistics(VmaTotalStatistics* pStats)
15124	{
15125	// Initialize.
15126	VmaClearDetailedStatistics(pStats->total);
15127	for(uint32_t i = `0`; i < VK_MAX_MEMORY_TYPES; ++i)
15128	VmaClearDetailedStatistics(pStats->memoryType[i]);
15129	for(uint32_t i = `0`; i < VK_MAX_MEMORY_HEAPS; ++i)
15130	VmaClearDetailedStatistics(pStats->memoryHeap[i]);
15131
15132	// Process default pools.
15133	for(uint32_t memTypeIndex = `0`; memTypeIndex < GetMemoryTypeCount(); ++memTypeIndex)
15134	{
15135	VmaBlockVector* const pBlockVector = m_pBlockVectors[memTypeIndex];
15136	if (pBlockVector != VMA_NULL)
15137	pBlockVector->AddDetailedStatistics(pStats->memoryType[memTypeIndex]);
15138	}
15139
15140	// Process custom pools.
15141	{
15142	VmaMutexLockRead lock(m_PoolsMutex, m_UseMutex);
15143	for(VmaPool pool = m_Pools.Front(); pool != VMA_NULL; pool = m_Pools.GetNext(pool))
15144	{
15145	VmaBlockVector& blockVector = pool->m_BlockVector;
15146	const uint32_t memTypeIndex = blockVector.GetMemoryTypeIndex();
15147	blockVector.AddDetailedStatistics(pStats->memoryType[memTypeIndex]);
15148	pool->m_DedicatedAllocations.AddDetailedStatistics(pStats->memoryType[memTypeIndex]);
15149	}
15150	}
15151
15152	// Process dedicated allocations.
15153	for(uint32_t memTypeIndex = `0`; memTypeIndex < GetMemoryTypeCount(); ++memTypeIndex)
15154	{
15155	m_DedicatedAllocations[memTypeIndex].AddDetailedStatistics(pStats->memoryType[memTypeIndex]);
15156	}
15157
15158	// Sum from memory types to memory heaps.
15159	for(uint32_t memTypeIndex = `0`; memTypeIndex < GetMemoryTypeCount(); ++memTypeIndex)
15160	{
15161	const uint32_t memHeapIndex = m_MemProps.memoryTypes[memTypeIndex].heapIndex;
15162	VmaAddDetailedStatistics(pStats->memoryHeap[memHeapIndex], pStats->memoryType[memTypeIndex]);
15163	}
15164
15165	// Sum from memory heaps to total.
15166	for(uint32_t memHeapIndex = `0`; memHeapIndex < GetMemoryHeapCount(); ++memHeapIndex)
15167	VmaAddDetailedStatistics(pStats->total, pStats->memoryHeap[memHeapIndex]);
15168
15169	VMA_ASSERT(pStats->total.statistics.allocationCount == `0` \|\|
15170	pStats->total.allocationSizeMax >= pStats->total.allocationSizeMin);
15171	VMA_ASSERT(pStats->total.unusedRangeCount == `0` \|\|
15172	pStats->total.unusedRangeSizeMax >= pStats->total.unusedRangeSizeMin);
15173	}
15174
15175	void VmaAllocator_T::GetHeapBudgets(VmaBudget* outBudgets, uint32_t firstHeap, uint32_t heapCount)
15176	{
15177	#if VMA_MEMORY_BUDGET
15178	if(m_UseExtMemoryBudget)
15179	{
15180	if(m_Budget.m_OperationsSinceBudgetFetch < `30`)
15181	{
15182	VmaMutexLockRead lockRead(m_Budget.m_BudgetMutex, m_UseMutex);
15183	for(uint32_t i = `0`; i < heapCount; ++i, ++outBudgets)
15184	{
15185	const uint32_t heapIndex = firstHeap + i;
15186
15187	outBudgets->statistics.blockCount = m_Budget.m_BlockCount[heapIndex];
15188	outBudgets->statistics.allocationCount = m_Budget.m_AllocationCount[heapIndex];
15189	outBudgets->statistics.blockBytes = m_Budget.m_BlockBytes[heapIndex];
15190	outBudgets->statistics.allocationBytes = m_Budget.m_AllocationBytes[heapIndex];
15191
15192	if(m_Budget.m_VulkanUsage[heapIndex] + outBudgets->statistics.blockBytes > m_Budget.m_BlockBytesAtBudgetFetch[heapIndex])
15193	{
15194	outBudgets->usage = m_Budget.m_VulkanUsage[heapIndex] +
15195	outBudgets->statistics.blockBytes - m_Budget.m_BlockBytesAtBudgetFetch[heapIndex];
15196	}
15197	else
15198	{
15199	outBudgets->usage = `0`;
15200	}
15201
15202	// Have to take MIN with heap size because explicit HeapSizeLimit is included in it.
15203	outBudgets->budget = VMA_MIN(
15204	m_Budget.m_VulkanBudget[heapIndex], m_MemProps.memoryHeaps[heapIndex].size);
15205	}
15206	}
15207	else
15208	{
15209	UpdateVulkanBudget(); // Outside of mutex lock
15210	GetHeapBudgets(outBudgets, firstHeap, heapCount); // Recursion
15211	}
15212	}
15213	else
15214	#endif
15215	{
15216	for(uint32_t i = `0`; i < heapCount; ++i, ++outBudgets)
15217	{
15218	const uint32_t heapIndex = firstHeap + i;
15219
15220	outBudgets->statistics.blockCount = m_Budget.m_BlockCount[heapIndex];
15221	outBudgets->statistics.allocationCount = m_Budget.m_AllocationCount[heapIndex];
15222	outBudgets->statistics.blockBytes = m_Budget.m_BlockBytes[heapIndex];
15223	outBudgets->statistics.allocationBytes = m_Budget.m_AllocationBytes[heapIndex];
15224
15225	outBudgets->usage = outBudgets->statistics.blockBytes;
15226	outBudgets->budget = m_MemProps.memoryHeaps[heapIndex].size * `8` / `10`; // 80% heuristics.
15227	}
15228	}
15229	}
15230
15231	void VmaAllocator_T::GetAllocationInfo(VmaAllocation hAllocation, VmaAllocationInfo* pAllocationInfo)
15232	{
15233	pAllocationInfo->memoryType = hAllocation->GetMemoryTypeIndex();
15234	pAllocationInfo->deviceMemory = hAllocation->GetMemory();
15235	pAllocationInfo->offset = hAllocation->GetOffset();
15236	pAllocationInfo->size = hAllocation->GetSize();
15237	pAllocationInfo->pMappedData = hAllocation->GetMappedData();
15238	pAllocationInfo->pUserData = hAllocation->GetUserData();
15239	pAllocationInfo->pName = hAllocation->GetName();
15240	}
15241
15242	VkResult VmaAllocator_T::CreatePool(const VmaPoolCreateInfo* pCreateInfo, VmaPool* pPool)
15243	{
15244	VMA_DEBUG_LOG(" CreatePool: MemoryTypeIndex=%u, flags=%u", pCreateInfo->memoryTypeIndex, pCreateInfo->flags);
15245
15246	VmaPoolCreateInfo newCreateInfo = *pCreateInfo;
15247
15248	// Protection against uninitialized new structure member. If garbage data are left there, this pointer dereference would crash.
15249	if(pCreateInfo->pMemoryAllocateNext)
15250	{
15251	VMA_ASSERT(((const VkBaseInStructure*)pCreateInfo->pMemoryAllocateNext)->sType != `0`);
15252	}
15253
15254	if(newCreateInfo.maxBlockCount == `0`)
15255	{
15256	newCreateInfo.maxBlockCount = SIZE_MAX;
15257	}
15258	if(newCreateInfo.minBlockCount > newCreateInfo.maxBlockCount)
15259	{
15260	return VK_ERROR_INITIALIZATION_FAILED;
15261	}
15262	// Memory type index out of range or forbidden.
15263	if(pCreateInfo->memoryTypeIndex >= GetMemoryTypeCount() \|\|
15264	((`1u` << pCreateInfo->memoryTypeIndex) & m_GlobalMemoryTypeBits) == `0`)
15265	{
15266	return VK_ERROR_FEATURE_NOT_PRESENT;
15267	}
15268	if(newCreateInfo.minAllocationAlignment > `0`)
15269	{
15270	VMA_ASSERT(VmaIsPow2(newCreateInfo.minAllocationAlignment));
15271	}
15272
15273	const VkDeviceSize preferredBlockSize = CalcPreferredBlockSize(newCreateInfo.memoryTypeIndex);
15274
15275	pPool = vma_new(this, VmaPool_T)(this*, newCreateInfo, preferredBlockSize);
15276
15277	VkResult res = (*pPool)->m_BlockVector.CreateMinBlocks();
15278	if(res != VK_SUCCESS)
15279	{
15280	vma_delete(this, *pPool);
15281	*pPool = VMA_NULL;
15282	return res;
15283	}
15284
15285	// Add to m_Pools.
15286	{
15287	VmaMutexLockWrite lock(m_PoolsMutex, m_UseMutex);
15288	(*pPool)->SetId(m_NextPoolId++);
15289	m_Pools.PushBack(*pPool);
15290	}
15291
15292	return VK_SUCCESS;
15293	}
15294
15295	void VmaAllocator_T::DestroyPool(VmaPool pool)
15296	{
15297	// Remove from m_Pools.
15298	{
15299	VmaMutexLockWrite lock(m_PoolsMutex, m_UseMutex);
15300	m_Pools.Remove(pool);
15301	}
15302
15303	vma_delete(this, pool);
15304	}
15305
15306	void VmaAllocator_T::GetPoolStatistics(VmaPool pool, VmaStatistics* pPoolStats)
15307	{
15308	VmaClearStatistics(*pPoolStats);
15309	pool->m_BlockVector.AddStatistics(*pPoolStats);
15310	pool->m_DedicatedAllocations.AddStatistics(*pPoolStats);
15311	}
15312
15313	void VmaAllocator_T::CalculatePoolStatistics(VmaPool pool, VmaDetailedStatistics* pPoolStats)
15314	{
15315	VmaClearDetailedStatistics(*pPoolStats);
15316	pool->m_BlockVector.AddDetailedStatistics(*pPoolStats);
15317	pool->m_DedicatedAllocations.AddDetailedStatistics(*pPoolStats);
15318	}
15319
15320	void VmaAllocator_T::SetCurrentFrameIndex(uint32_t frameIndex)
15321	{
15322	m_CurrentFrameIndex.store(frameIndex);
15323
15324	#if VMA_MEMORY_BUDGET
15325	if(m_UseExtMemoryBudget)
15326	{
15327	UpdateVulkanBudget();
15328	}
15329	#endif // #if VMA_MEMORY_BUDGET
15330	}
15331
15332	VkResult VmaAllocator_T::CheckPoolCorruption(VmaPool hPool)
15333	{
15334	return hPool->m_BlockVector.CheckCorruption();
15335	}
15336
15337	VkResult VmaAllocator_T::CheckCorruption(uint32_t memoryTypeBits)
15338	{
15339	VkResult finalRes = VK_ERROR_FEATURE_NOT_PRESENT;
15340
15341	// Process default pools.
15342	for(uint32_t memTypeIndex = `0`; memTypeIndex < GetMemoryTypeCount(); ++memTypeIndex)
15343	{
15344	VmaBlockVector* const pBlockVector = m_pBlockVectors[memTypeIndex];
15345	if(pBlockVector != VMA_NULL)
15346	{
15347	VkResult localRes = pBlockVector->CheckCorruption();
15348	switch(localRes)
15349	{
15350	case VK_ERROR_FEATURE_NOT_PRESENT:
15351	break;
15352	case VK_SUCCESS:
15353	finalRes = VK_SUCCESS;
15354	break;
15355	default:
15356	return localRes;
15357	}
15358	}
15359	}
15360
15361	// Process custom pools.
15362	{
15363	VmaMutexLockRead lock(m_PoolsMutex, m_UseMutex);
15364	for(VmaPool pool = m_Pools.Front(); pool != VMA_NULL; pool = m_Pools.GetNext(pool))
15365	{
15366	if(((`1u` << pool->m_BlockVector.GetMemoryTypeIndex()) & memoryTypeBits) != `0`)
15367	{
15368	VkResult localRes = pool->m_BlockVector.CheckCorruption();
15369	switch(localRes)
15370	{
15371	case VK_ERROR_FEATURE_NOT_PRESENT:
15372	break;
15373	case VK_SUCCESS:
15374	finalRes = VK_SUCCESS;
15375	break;
15376	default:
15377	return localRes;
15378	}
15379	}
15380	}
15381	}
15382
15383	return finalRes;
15384	}
15385
15386	VkResult VmaAllocator_T::AllocateVulkanMemory(const VkMemoryAllocateInfo* pAllocateInfo, VkDeviceMemory* pMemory)
15387	{
15388	AtomicTransactionalIncrement<uint32_t> deviceMemoryCountIncrement;
15389	const uint64_t prevDeviceMemoryCount = deviceMemoryCountIncrement.Increment(&m_DeviceMemoryCount);
15390	#if VMA_DEBUG_DONT_EXCEED_MAX_MEMORY_ALLOCATION_COUNT
15391	if(prevDeviceMemoryCount >= m_PhysicalDeviceProperties.limits.maxMemoryAllocationCount)
15392	{
15393	return VK_ERROR_TOO_MANY_OBJECTS;
15394	}
15395	#endif
15396
15397	const uint32_t heapIndex = MemoryTypeIndexToHeapIndex(pAllocateInfo->memoryTypeIndex);
15398
15399	// HeapSizeLimit is in effect for this heap.
15400	if((m_HeapSizeLimitMask & (`1u` << heapIndex)) != `0`)
15401	{
15402	const VkDeviceSize heapSize = m_MemProps.memoryHeaps[heapIndex].size;
15403	VkDeviceSize blockBytes = m_Budget.m_BlockBytes[heapIndex];
15404	for(;;)
15405	{
15406	const VkDeviceSize blockBytesAfterAllocation = blockBytes + pAllocateInfo->allocationSize;
15407	if(blockBytesAfterAllocation > heapSize)
15408	{
15409	return VK_ERROR_OUT_OF_DEVICE_MEMORY;
15410	}
15411	if(m_Budget.m_BlockBytes[heapIndex].compare_exchange_strong(blockBytes, blockBytesAfterAllocation))
15412	{
15413	break;
15414	}
15415	}
15416	}
15417	else
15418	{
15419	m_Budget.m_BlockBytes[heapIndex] += pAllocateInfo->allocationSize;
15420	}
15421	++m_Budget.m_BlockCount[heapIndex];
15422
15423	// VULKAN CALL vkAllocateMemory.
15424	VkResult res = (*m_VulkanFunctions.vkAllocateMemory)(m_hDevice, pAllocateInfo, GetAllocationCallbacks(), pMemory);
15425
15426	if(res == VK_SUCCESS)
15427	{
15428	#if VMA_MEMORY_BUDGET
15429	++m_Budget.m_OperationsSinceBudgetFetch;
15430	#endif
15431
15432	// Informative callback.
15433	if(m_DeviceMemoryCallbacks.pfnAllocate != VMA_NULL)
15434	{
15435	(m_DeviceMemoryCallbacks.pfnAllocate)(this, pAllocateInfo->memoryTypeIndex, pMemory, pAllocateInfo->allocationSize, m_DeviceMemoryCallbacks.pUserData);
15436	}
15437
15438	deviceMemoryCountIncrement.Commit();
15439	}
15440	else
15441	{
15442	--m_Budget.m_BlockCount[heapIndex];
15443	m_Budget.m_BlockBytes[heapIndex] -= pAllocateInfo->allocationSize;
15444	}
15445
15446	return res;
15447	}
15448
15449	void VmaAllocator_T::FreeVulkanMemory(uint32_t memoryType, VkDeviceSize size, VkDeviceMemory hMemory)
15450	{
15451	// Informative callback.
15452	if(m_DeviceMemoryCallbacks.pfnFree != VMA_NULL)
15453	{
15454	(m_DeviceMemoryCallbacks.pfnFree)(this*, memoryType, hMemory, size, m_DeviceMemoryCallbacks.pUserData);
15455	}
15456
15457	// VULKAN CALL vkFreeMemory.
15458	(*m_VulkanFunctions.vkFreeMemory)(m_hDevice, hMemory, GetAllocationCallbacks());
15459
15460	const uint32_t heapIndex = MemoryTypeIndexToHeapIndex(memoryType);
15461	--m_Budget.m_BlockCount[heapIndex];
15462	m_Budget.m_BlockBytes[heapIndex] -= size;
15463
15464	--m_DeviceMemoryCount;
15465	}
15466
15467	VkResult VmaAllocator_T::BindVulkanBuffer(
15468	VkDeviceMemory memory,
15469	VkDeviceSize memoryOffset,
15470	VkBuffer buffer,
15471	const void* pNext)
15472	{
15473	if(pNext != VMA_NULL)
15474	{
15475	#if VMA_VULKAN_VERSION >= 1001000 \|\| VMA_BIND_MEMORY2
15476	if((m_UseKhrBindMemory2 \|\| m_VulkanApiVersion >= VK_MAKE_VERSION(`1`, `1`, `0`)) &&
15477	m_VulkanFunctions.vkBindBufferMemory2KHR != VMA_NULL)
15478	{
15479	VkBindBufferMemoryInfoKHR bindBufferMemoryInfo = { VK_STRUCTURE_TYPE_BIND_BUFFER_MEMORY_INFO_KHR };
15480	bindBufferMemoryInfo.pNext = pNext;
15481	bindBufferMemoryInfo.buffer = buffer;
15482	bindBufferMemoryInfo.memory = memory;
15483	bindBufferMemoryInfo.memoryOffset = memoryOffset;
15484	return (*m_VulkanFunctions.vkBindBufferMemory2KHR)(m_hDevice, `1`, &bindBufferMemoryInfo);
15485	}
15486	else
15487	#endif // #if VMA_VULKAN_VERSION >= 1001000 \|\| VMA_BIND_MEMORY2
15488	{
15489	return VK_ERROR_EXTENSION_NOT_PRESENT;
15490	}
15491	}
15492	else
15493	{
15494	return (*m_VulkanFunctions.vkBindBufferMemory)(m_hDevice, buffer, memory, memoryOffset);
15495	}
15496	}
15497
15498	VkResult VmaAllocator_T::BindVulkanImage(
15499	VkDeviceMemory memory,
15500	VkDeviceSize memoryOffset,
15501	VkImage image,
15502	const void* pNext)
15503	{
15504	if(pNext != VMA_NULL)
15505	{
15506	#if VMA_VULKAN_VERSION >= 1001000 \|\| VMA_BIND_MEMORY2
15507	if((m_UseKhrBindMemory2 \|\| m_VulkanApiVersion >= VK_MAKE_VERSION(`1`, `1`, `0`)) &&
15508	m_VulkanFunctions.vkBindImageMemory2KHR != VMA_NULL)
15509	{
15510	VkBindImageMemoryInfoKHR bindBufferMemoryInfo = { VK_STRUCTURE_TYPE_BIND_IMAGE_MEMORY_INFO_KHR };
15511	bindBufferMemoryInfo.pNext = pNext;
15512	bindBufferMemoryInfo.image = image;
15513	bindBufferMemoryInfo.memory = memory;
15514	bindBufferMemoryInfo.memoryOffset = memoryOffset;
15515	return (*m_VulkanFunctions.vkBindImageMemory2KHR)(m_hDevice, `1`, &bindBufferMemoryInfo);
15516	}
15517	else
15518	#endif // #if VMA_BIND_MEMORY2
15519	{
15520	return VK_ERROR_EXTENSION_NOT_PRESENT;
15521	}
15522	}
15523	else
15524	{
15525	return (*m_VulkanFunctions.vkBindImageMemory)(m_hDevice, image, memory, memoryOffset);
15526	}
15527	}
15528
15529	VkResult VmaAllocator_T::Map(VmaAllocation hAllocation, void** ppData)
15530	{
15531	switch(hAllocation->GetType())
15532	{
15533	case VmaAllocation_T::ALLOCATION_TYPE_BLOCK:
15534	{
15535	VmaDeviceMemoryBlock* const pBlock = hAllocation->GetBlock();
15536	char *pBytes = VMA_NULL;
15537	VkResult res = pBlock->Map(this, `1`, (void**)&pBytes);
15538	if(res == VK_SUCCESS)
15539	{
15540	*ppData = pBytes + (ptrdiff_t)hAllocation->GetOffset();
15541	hAllocation->BlockAllocMap();
15542	}
15543	return res;
15544	}
15545	case VmaAllocation_T::ALLOCATION_TYPE_DEDICATED:
15546	return hAllocation->DedicatedAllocMap(this, ppData);
15547	default:
15548	VMA_ASSERT(`0`);
15549	return VK_ERROR_MEMORY_MAP_FAILED;
15550	}
15551	}
15552
15553	void VmaAllocator_T::Unmap(VmaAllocation hAllocation)
15554	{
15555	switch(hAllocation->GetType())
15556	{
15557	case VmaAllocation_T::ALLOCATION_TYPE_BLOCK:
15558	{
15559	VmaDeviceMemoryBlock* const pBlock = hAllocation->GetBlock();
15560	hAllocation->BlockAllocUnmap();
15561	pBlock->Unmap(this, `1`);
15562	}
15563	break;
15564	case VmaAllocation_T::ALLOCATION_TYPE_DEDICATED:
15565	hAllocation->DedicatedAllocUnmap(this);
15566	break;
15567	default:
15568	VMA_ASSERT(`0`);
15569	}
15570	}
15571
15572	VkResult VmaAllocator_T::BindBufferMemory(
15573	VmaAllocation hAllocation,
15574	VkDeviceSize allocationLocalOffset,
15575	VkBuffer hBuffer,
15576	const void* pNext)
15577	{
15578	VkResult res = VK_SUCCESS;
15579	switch(hAllocation->GetType())
15580	{
15581	case VmaAllocation_T::ALLOCATION_TYPE_DEDICATED:
15582	res = BindVulkanBuffer(hAllocation->GetMemory(), allocationLocalOffset, hBuffer, pNext);
15583	break;
15584	case VmaAllocation_T::ALLOCATION_TYPE_BLOCK:
15585	{
15586	VmaDeviceMemoryBlock* const pBlock = hAllocation->GetBlock();
15587	VMA_ASSERT(pBlock && "Binding buffer to allocation that doesn't belong to any block.");
15588	res = pBlock->BindBufferMemory(this, hAllocation, allocationLocalOffset, hBuffer, pNext);
15589	break;
15590	}
15591	default:
15592	VMA_ASSERT(`0`);
15593	}
15594	return res;
15595	}
15596
15597	VkResult VmaAllocator_T::BindImageMemory(
15598	VmaAllocation hAllocation,
15599	VkDeviceSize allocationLocalOffset,
15600	VkImage hImage,
15601	const void* pNext)
15602	{
15603	VkResult res = VK_SUCCESS;
15604	switch(hAllocation->GetType())
15605	{
15606	case VmaAllocation_T::ALLOCATION_TYPE_DEDICATED:
15607	res = BindVulkanImage(hAllocation->GetMemory(), allocationLocalOffset, hImage, pNext);
15608	break;
15609	case VmaAllocation_T::ALLOCATION_TYPE_BLOCK:
15610	{
15611	VmaDeviceMemoryBlock* pBlock = hAllocation->GetBlock();
15612	VMA_ASSERT(pBlock && "Binding image to allocation that doesn't belong to any block.");
15613	res = pBlock->BindImageMemory(this, hAllocation, allocationLocalOffset, hImage, pNext);
15614	break;
15615	}
15616	default:
15617	VMA_ASSERT(`0`);
15618	}
15619	return res;
15620	}
15621
15622	VkResult VmaAllocator_T::FlushOrInvalidateAllocation(
15623	VmaAllocation hAllocation,
15624	VkDeviceSize offset, VkDeviceSize size,
15625	VMA_CACHE_OPERATION op)
15626	{
15627	VkResult res = VK_SUCCESS;
15628
15629	VkMappedMemoryRange memRange = {};
15630	if(GetFlushOrInvalidateRange(hAllocation, offset, size, memRange))
15631	{
15632	switch(op)
15633	{
15634	case VMA_CACHE_FLUSH:
15635	res = (*GetVulkanFunctions().vkFlushMappedMemoryRanges)(m_hDevice, `1`, &memRange);
15636	break;
15637	case VMA_CACHE_INVALIDATE:
15638	res = (*GetVulkanFunctions().vkInvalidateMappedMemoryRanges)(m_hDevice, `1`, &memRange);
15639	break;
15640	default:
15641	VMA_ASSERT(`0`);
15642	}
15643	}
15644	// else: Just ignore this call.
15645	return res;
15646	}
15647
15648	VkResult VmaAllocator_T::FlushOrInvalidateAllocations(
15649	uint32_t allocationCount,
15650	const VmaAllocation* allocations,
15651	const VkDeviceSize* offsets, const VkDeviceSize* sizes,
15652	VMA_CACHE_OPERATION op)
15653	{
15654	typedef VmaStlAllocator<VkMappedMemoryRange> RangeAllocator;
15655	typedef VmaSmallVector<VkMappedMemoryRange, RangeAllocator, `16`> RangeVector;
15656	RangeVector ranges = RangeVector(RangeAllocator(GetAllocationCallbacks()));
15657
15658	for(uint32_t allocIndex = `0`; allocIndex < allocationCount; ++allocIndex)
15659	{
15660	const VmaAllocation alloc = allocations[allocIndex];
15661	const VkDeviceSize offset = offsets != VMA_NULL ? offsets[allocIndex] : `0`;
15662	const VkDeviceSize size = sizes != VMA_NULL ? sizes[allocIndex] : VK_WHOLE_SIZE;
15663	VkMappedMemoryRange newRange;
15664	if(GetFlushOrInvalidateRange(alloc, offset, size, newRange))
15665	{
15666	ranges.push_back(newRange);
15667	}
15668	}
15669
15670	VkResult res = VK_SUCCESS;
15671	if(!ranges.empty())
15672	{
15673	switch(op)
15674	{
15675	case VMA_CACHE_FLUSH:
15676	res = (*GetVulkanFunctions().vkFlushMappedMemoryRanges)(m_hDevice, (uint32_t)ranges.size(), ranges.data());
15677	break;
15678	case VMA_CACHE_INVALIDATE:
15679	res = (*GetVulkanFunctions().vkInvalidateMappedMemoryRanges)(m_hDevice, (uint32_t)ranges.size(), ranges.data());
15680	break;
15681	default:
15682	VMA_ASSERT(`0`);
15683	}
15684	}
15685	// else: Just ignore this call.
15686	return res;
15687	}
15688
15689	void VmaAllocator_T::FreeDedicatedMemory(const VmaAllocation allocation)
15690	{
15691	VMA_ASSERT(allocation && allocation->GetType() == VmaAllocation_T::ALLOCATION_TYPE_DEDICATED);
15692
15693	const uint32_t memTypeIndex = allocation->GetMemoryTypeIndex();
15694	VmaPool parentPool = allocation->GetParentPool();
15695	if(parentPool == VK_NULL_HANDLE)
15696	{
15697	// Default pool
15698	m_DedicatedAllocations[memTypeIndex].Unregister(allocation);
15699	}
15700	else
15701	{
15702	// Custom pool
15703	parentPool->m_DedicatedAllocations.Unregister(allocation);
15704	}
15705
15706	VkDeviceMemory hMemory = allocation->GetMemory();
15707
15708	/*
15709	There is no need to call this, because Vulkan spec allows to skip vkUnmapMemory
15710	before vkFreeMemory.
15711
15712	if(allocation->GetMappedData() != VMA_NULL)
15713	{
15714	(m_VulkanFunctions.vkUnmapMemory)(m_hDevice, hMemory);*
15715	}
15716	*/
15717
15718	FreeVulkanMemory(memTypeIndex, allocation->GetSize(), hMemory);
15719
15720	m_Budget.RemoveAllocation(MemoryTypeIndexToHeapIndex(allocation->GetMemoryTypeIndex()), allocation->GetSize());
15721	m_AllocationObjectAllocator.Free(allocation);
15722
15723	VMA_DEBUG_LOG(" Freed DedicatedMemory MemoryTypeIndex=%u", memTypeIndex);
15724	}
15725
15726	uint32_t VmaAllocator_T::CalculateGpuDefragmentationMemoryTypeBits() const
15727	{
15728	VkBufferCreateInfo dummyBufCreateInfo;
15729	VmaFillGpuDefragmentationBufferCreateInfo(dummyBufCreateInfo);
15730
15731	uint32_t memoryTypeBits = `0`;
15732
15733	// Create buffer.
15734	VkBuffer buf = VK_NULL_HANDLE;
15735	VkResult res = (*GetVulkanFunctions().vkCreateBuffer)(
15736	m_hDevice, &dummyBufCreateInfo, GetAllocationCallbacks(), &buf);
15737	if(res == VK_SUCCESS)
15738	{
15739	// Query for supported memory types.
15740	VkMemoryRequirements memReq;
15741	(*GetVulkanFunctions().vkGetBufferMemoryRequirements)(m_hDevice, buf, &memReq);
15742	memoryTypeBits = memReq.memoryTypeBits;
15743
15744	// Destroy buffer.
15745	(*GetVulkanFunctions().vkDestroyBuffer)(m_hDevice, buf, GetAllocationCallbacks());
15746	}
15747
15748	return memoryTypeBits;
15749	}
15750
15751	uint32_t VmaAllocator_T::CalculateGlobalMemoryTypeBits() const
15752	{
15753	// Make sure memory information is already fetched.
15754	VMA_ASSERT(GetMemoryTypeCount() > `0`);
15755
15756	uint32_t memoryTypeBits = UINT32_MAX;
15757
15758	if(!m_UseAmdDeviceCoherentMemory)
15759	{
15760	// Exclude memory types that have VK_MEMORY_PROPERTY_DEVICE_COHERENT_BIT_AMD.
15761	for(uint32_t memTypeIndex = `0`; memTypeIndex < GetMemoryTypeCount(); ++memTypeIndex)
15762	{
15763	if((m_MemProps.memoryTypes[memTypeIndex].propertyFlags & VK_MEMORY_PROPERTY_DEVICE_COHERENT_BIT_AMD_COPY) != `0`)
15764	{
15765	memoryTypeBits &= ~(`1u` << memTypeIndex);
15766	}
15767	}
15768	}
15769
15770	return memoryTypeBits;
15771	}
15772
15773	bool VmaAllocator_T::GetFlushOrInvalidateRange(
15774	VmaAllocation allocation,
15775	VkDeviceSize offset, VkDeviceSize size,
15776	VkMappedMemoryRange& outRange) const
15777	{
15778	const uint32_t memTypeIndex = allocation->GetMemoryTypeIndex();
15779	if(size > `0` && IsMemoryTypeNonCoherent(memTypeIndex))
15780	{
15781	const VkDeviceSize nonCoherentAtomSize = m_PhysicalDeviceProperties.limits.nonCoherentAtomSize;
15782	const VkDeviceSize allocationSize = allocation->GetSize();
15783	VMA_ASSERT(offset <= allocationSize);
15784
15785	outRange.sType = VK_STRUCTURE_TYPE_MAPPED_MEMORY_RANGE;
15786	outRange.pNext = VMA_NULL;
15787	outRange.memory = allocation->GetMemory();
15788
15789	switch(allocation->GetType())
15790	{
15791	case VmaAllocation_T::ALLOCATION_TYPE_DEDICATED:
15792	outRange.offset = VmaAlignDown(offset, nonCoherentAtomSize);
15793	if(size == VK_WHOLE_SIZE)
15794	{
15795	outRange.size = allocationSize - outRange.offset;
15796	}
15797	else
15798	{
15799	VMA_ASSERT(offset + size <= allocationSize);
15800	outRange.size = VMA_MIN(
15801	VmaAlignUp(size + (offset - outRange.offset), nonCoherentAtomSize),
15802	allocationSize - outRange.offset);
15803	}
15804	break;
15805	case VmaAllocation_T::ALLOCATION_TYPE_BLOCK:
15806	{
15807	// 1. Still within this allocation.
15808	outRange.offset = VmaAlignDown(offset, nonCoherentAtomSize);
15809	if(size == VK_WHOLE_SIZE)
15810	{
15811	size = allocationSize - offset;
15812	}
15813	else
15814	{
15815	VMA_ASSERT(offset + size <= allocationSize);
15816	}
15817	outRange.size = VmaAlignUp(size + (offset - outRange.offset), nonCoherentAtomSize);
15818
15819	// 2. Adjust to whole block.
15820	const VkDeviceSize allocationOffset = allocation->GetOffset();
15821	VMA_ASSERT(allocationOffset % nonCoherentAtomSize == `0`);
15822	const VkDeviceSize blockSize = allocation->GetBlock()->m_pMetadata->GetSize();
15823	outRange.offset += allocationOffset;
15824	outRange.size = VMA_MIN(outRange.size, blockSize - outRange.offset);
15825
15826	break;
15827	}
15828	default:
15829	VMA_ASSERT(`0`);
15830	}
15831	return true;
15832	}
15833	return false;
15834	}
15835
15836	#if VMA_MEMORY_BUDGET
15837	void VmaAllocator_T::UpdateVulkanBudget()
15838	{
15839	VMA_ASSERT(m_UseExtMemoryBudget);
15840
15841	VkPhysicalDeviceMemoryProperties2KHR memProps = { VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_MEMORY_PROPERTIES_2_KHR };
15842
15843	VkPhysicalDeviceMemoryBudgetPropertiesEXT budgetProps = { VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_MEMORY_BUDGET_PROPERTIES_EXT };
15844	VmaPnextChainPushFront(&memProps, &budgetProps);
15845
15846	GetVulkanFunctions().vkGetPhysicalDeviceMemoryProperties2KHR(m_PhysicalDevice, &memProps);
15847
15848	{
15849	VmaMutexLockWrite lockWrite(m_Budget.m_BudgetMutex, m_UseMutex);
15850
15851	for(uint32_t heapIndex = `0`; heapIndex < GetMemoryHeapCount(); ++heapIndex)
15852	{
15853	m_Budget.m_VulkanUsage[heapIndex] = budgetProps.heapUsage[heapIndex];
15854	m_Budget.m_VulkanBudget[heapIndex] = budgetProps.heapBudget[heapIndex];
15855	m_Budget.m_BlockBytesAtBudgetFetch[heapIndex] = m_Budget.m_BlockBytes[heapIndex].load();
15856
15857	// Some bugged drivers return the budget incorrectly, e.g. 0 or much bigger than heap size.
15858	if(m_Budget.m_VulkanBudget[heapIndex] == `0`)
15859	{
15860	m_Budget.m_VulkanBudget[heapIndex] = m_MemProps.memoryHeaps[heapIndex].size * `8` / `10`; // 80% heuristics.
15861	}
15862	else if(m_Budget.m_VulkanBudget[heapIndex] > m_MemProps.memoryHeaps[heapIndex].size)
15863	{
15864	m_Budget.m_VulkanBudget[heapIndex] = m_MemProps.memoryHeaps[heapIndex].size;
15865	}
15866	if(m_Budget.m_VulkanUsage[heapIndex] == `0` && m_Budget.m_BlockBytesAtBudgetFetch[heapIndex] > `0`)
15867	{
15868	m_Budget.m_VulkanUsage[heapIndex] = m_Budget.m_BlockBytesAtBudgetFetch[heapIndex];
15869	}
15870	}
15871	m_Budget.m_OperationsSinceBudgetFetch = `0`;
15872	}
15873	}
15874	#endif // VMA_MEMORY_BUDGET
15875
15876	void VmaAllocator_T::FillAllocation(const VmaAllocation hAllocation, uint8_t pattern)
15877	{
15878	if(VMA_DEBUG_INITIALIZE_ALLOCATIONS &&
15879	hAllocation->IsMappingAllowed() &&
15880	(m_MemProps.memoryTypes[hAllocation->GetMemoryTypeIndex()].propertyFlags & VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT) != `0`)
15881	{
15882	void* pData = VMA_NULL;
15883	VkResult res = Map(hAllocation, &pData);
15884	if(res == VK_SUCCESS)
15885	{
15886	memset(pData, (int)pattern, (size_t)hAllocation->GetSize());
15887	FlushOrInvalidateAllocation(hAllocation, `0`, VK_WHOLE_SIZE, VMA_CACHE_FLUSH);
15888	Unmap(hAllocation);
15889	}
15890	else
15891	{
15892	VMA_ASSERT(`0` && "VMA_DEBUG_INITIALIZE_ALLOCATIONS is enabled, but couldn't map memory to fill allocation.");
15893	}
15894	}
15895	}
15896
15897	uint32_t VmaAllocator_T::GetGpuDefragmentationMemoryTypeBits()
15898	{
15899	uint32_t memoryTypeBits = m_GpuDefragmentationMemoryTypeBits.load();
15900	if(memoryTypeBits == UINT32_MAX)
15901	{
15902	memoryTypeBits = CalculateGpuDefragmentationMemoryTypeBits();
15903	m_GpuDefragmentationMemoryTypeBits.store(memoryTypeBits);
15904	}
15905	return memoryTypeBits;
15906	}
15907
15908	#if VMA_STATS_STRING_ENABLED
15909	void VmaAllocator_T::PrintDetailedMap(VmaJsonWriter& json)
15910	{
15911	json.WriteString("DefaultPools");
15912	json.BeginObject();
15913	{
15914	for (uint32_t memTypeIndex = `0`; memTypeIndex < GetMemoryTypeCount(); ++memTypeIndex)
15915	{
15916	VmaBlockVector* pBlockVector = m_pBlockVectors[memTypeIndex];
15917	VmaDedicatedAllocationList& dedicatedAllocList = m_DedicatedAllocations[memTypeIndex];
15918	if (pBlockVector != VMA_NULL)
15919	{
15920	json.BeginString("Type ");
15921	json.ContinueString(memTypeIndex);
15922	json.EndString();
15923	json.BeginObject();
15924	{
15925	json.WriteString("PreferredBlockSize");
15926	json.WriteNumber(pBlockVector->GetPreferredBlockSize());
15927
15928	json.WriteString("Blocks");
15929	pBlockVector->PrintDetailedMap(json);
15930
15931	json.WriteString("DedicatedAllocations");
15932	dedicatedAllocList.BuildStatsString(json);
15933	}
15934	json.EndObject();
15935	}
15936	}
15937	}
15938	json.EndObject();
15939
15940	json.WriteString("CustomPools");
15941	json.BeginObject();
15942	{
15943	VmaMutexLockRead lock(m_PoolsMutex, m_UseMutex);
15944	if (!m_Pools.IsEmpty())
15945	{
15946	for (uint32_t memTypeIndex = `0`; memTypeIndex < GetMemoryTypeCount(); ++memTypeIndex)
15947	{
15948	bool displayType = true;
15949	size_t index = `0`;
15950	for (VmaPool pool = m_Pools.Front(); pool != VMA_NULL; pool = m_Pools.GetNext(pool))
15951	{
15952	VmaBlockVector& blockVector = pool->m_BlockVector;
15953	if (blockVector.GetMemoryTypeIndex() == memTypeIndex)
15954	{
15955	if (displayType)
15956	{
15957	json.BeginString("Type ");
15958	json.ContinueString(memTypeIndex);
15959	json.EndString();
15960	json.BeginArray();
15961	displayType = false;
15962	}
15963
15964	json.BeginObject();
15965	{
15966	json.WriteString("Name");
15967	json.BeginString();
15968	json.ContinueString_Size(index++);
15969	if (pool->GetName())
15970	{
15971	json.ContinueString(" - ");
15972	json.ContinueString(pool->GetName());
15973	}
15974	json.EndString();
15975
15976	json.WriteString("PreferredBlockSize");
15977	json.WriteNumber(blockVector.GetPreferredBlockSize());
15978
15979	json.WriteString("Blocks");
15980	blockVector.PrintDetailedMap(json);
15981
15982	json.WriteString("DedicatedAllocations");
15983	pool->m_DedicatedAllocations.BuildStatsString(json);
15984	}
15985	json.EndObject();
15986	}
15987	}
15988
15989	if (!displayType)
15990	json.EndArray();
15991	}
15992	}
15993	}
15994	json.EndObject();
15995	}
15996	#endif // VMA_STATS_STRING_ENABLED
15997	#endif // _VMA_ALLOCATOR_T_FUNCTIONS
15998
15999
16000	#ifndef _VMA_PUBLIC_INTERFACE
16001	VMA_CALL_PRE VkResult VMA_CALL_POST vmaCreateAllocator(
16002	const VmaAllocatorCreateInfo* pCreateInfo,
16003	VmaAllocator* pAllocator)
16004	{
16005	VMA_ASSERT(pCreateInfo && pAllocator);
16006	VMA_ASSERT(pCreateInfo->vulkanApiVersion == `0` \|\|
16007	(VK_VERSION_MAJOR(pCreateInfo->vulkanApiVersion) == `1` && VK_VERSION_MINOR(pCreateInfo->vulkanApiVersion) <= `3`));
16008	VMA_DEBUG_LOG("vmaCreateAllocator");
16009	*pAllocator = vma_new(pCreateInfo->pAllocationCallbacks, VmaAllocator_T)(pCreateInfo);
16010	VkResult result = (*pAllocator)->Init(pCreateInfo);
16011	if(result < `0`)
16012	{
16013	vma_delete(pCreateInfo->pAllocationCallbacks, *pAllocator);
16014	*pAllocator = VK_NULL_HANDLE;
16015	}
16016	return result;
16017	}
16018
16019	VMA_CALL_PRE void VMA_CALL_POST vmaDestroyAllocator(
16020	VmaAllocator allocator)
16021	{
16022	if(allocator != VK_NULL_HANDLE)
16023	{
16024	VMA_DEBUG_LOG("vmaDestroyAllocator");
16025	VkAllocationCallbacks allocationCallbacks = allocator->m_AllocationCallbacks; // Have to copy the callbacks when destroying.
16026	vma_delete(&allocationCallbacks, allocator);
16027	}
16028	}
16029
16030	VMA_CALL_PRE void VMA_CALL_POST vmaGetAllocatorInfo(VmaAllocator allocator, VmaAllocatorInfo* pAllocatorInfo)
16031	{
16032	VMA_ASSERT(allocator && pAllocatorInfo);
16033	pAllocatorInfo->instance = allocator->m_hInstance;
16034	pAllocatorInfo->physicalDevice = allocator->GetPhysicalDevice();
16035	pAllocatorInfo->device = allocator->m_hDevice;
16036	}
16037
16038	VMA_CALL_PRE void VMA_CALL_POST vmaGetPhysicalDeviceProperties(
16039	VmaAllocator allocator,
16040	const VkPhysicalDeviceProperties **ppPhysicalDeviceProperties)
16041	{
16042	VMA_ASSERT(allocator && ppPhysicalDeviceProperties);
16043	*ppPhysicalDeviceProperties = &allocator->m_PhysicalDeviceProperties;
16044	}
16045
16046	VMA_CALL_PRE void VMA_CALL_POST vmaGetMemoryProperties(
16047	VmaAllocator allocator,
16048	const VkPhysicalDeviceMemoryProperties** ppPhysicalDeviceMemoryProperties)
16049	{
16050	VMA_ASSERT(allocator && ppPhysicalDeviceMemoryProperties);
16051	*ppPhysicalDeviceMemoryProperties = &allocator->m_MemProps;
16052	}
16053
16054	VMA_CALL_PRE void VMA_CALL_POST vmaGetMemoryTypeProperties(
16055	VmaAllocator allocator,
16056	uint32_t memoryTypeIndex,
16057	VkMemoryPropertyFlags* pFlags)
16058	{
16059	VMA_ASSERT(allocator && pFlags);
16060	VMA_ASSERT(memoryTypeIndex < allocator->GetMemoryTypeCount());
16061	*pFlags = allocator->m_MemProps.memoryTypes[memoryTypeIndex].propertyFlags;
16062	}
16063
16064	VMA_CALL_PRE void VMA_CALL_POST vmaSetCurrentFrameIndex(
16065	VmaAllocator allocator,
16066	uint32_t frameIndex)
16067	{
16068	VMA_ASSERT(allocator);
16069
16070	VMA_DEBUG_GLOBAL_MUTEX_LOCK
16071
16072	allocator->SetCurrentFrameIndex(frameIndex);
16073	}
16074
16075	VMA_CALL_PRE void VMA_CALL_POST vmaCalculateStatistics(
16076	VmaAllocator allocator,
16077	VmaTotalStatistics* pStats)
16078	{
16079	VMA_ASSERT(allocator && pStats);
16080	VMA_DEBUG_GLOBAL_MUTEX_LOCK
16081	allocator->CalculateStatistics(pStats);
16082	}
16083
16084	VMA_CALL_PRE void VMA_CALL_POST vmaGetHeapBudgets(
16085	VmaAllocator allocator,
16086	VmaBudget* pBudgets)
16087	{
16088	VMA_ASSERT(allocator && pBudgets);
16089	VMA_DEBUG_GLOBAL_MUTEX_LOCK
16090	allocator->GetHeapBudgets(pBudgets, `0`, allocator->GetMemoryHeapCount());
16091	}
16092
16093	#if VMA_STATS_STRING_ENABLED
16094
16095	VMA_CALL_PRE void VMA_CALL_POST vmaBuildStatsString(
16096	VmaAllocator allocator,
16097	char** ppStatsString,
16098	VkBool32 detailedMap)
16099	{
16100	VMA_ASSERT(allocator && ppStatsString);
16101	VMA_DEBUG_GLOBAL_MUTEX_LOCK
16102
16103	VmaStringBuilder sb(allocator->GetAllocationCallbacks());
16104	{
16105	VmaBudget budgets[VK_MAX_MEMORY_HEAPS];
16106	allocator->GetHeapBudgets(budgets, `0`, allocator->GetMemoryHeapCount());
16107
16108	VmaTotalStatistics stats;
16109	allocator->CalculateStatistics(&stats);
16110
16111	VmaJsonWriter json(allocator->GetAllocationCallbacks(), sb);
16112	json.BeginObject();
16113	{
16114	json.WriteString("General");
16115	json.BeginObject();
16116	{
16117	const VkPhysicalDeviceProperties& deviceProperties = allocator->m_PhysicalDeviceProperties;
16118	const VkPhysicalDeviceMemoryProperties& memoryProperties = allocator->m_MemProps;
16119
16120	json.WriteString("API");
16121	json.WriteString("Vulkan");
16122
16123	json.WriteString("apiVersion");
16124	json.BeginString();
16125	json.ContinueString(VK_API_VERSION_MAJOR(deviceProperties.apiVersion));
16126	json.ContinueString(".");
16127	json.ContinueString(VK_API_VERSION_MINOR(deviceProperties.apiVersion));
16128	json.ContinueString(".");
16129	json.ContinueString(VK_API_VERSION_PATCH(deviceProperties.apiVersion));
16130	json.EndString();
16131
16132	json.WriteString("GPU");
16133	json.WriteString(deviceProperties.deviceName);
16134	json.WriteString("deviceType");
16135	json.WriteNumber(static_cast<uint32_t>(deviceProperties.deviceType));
16136
16137	json.WriteString("maxMemoryAllocationCount");
16138	json.WriteNumber(deviceProperties.limits.maxMemoryAllocationCount);
16139	json.WriteString("bufferImageGranularity");
16140	json.WriteNumber(deviceProperties.limits.bufferImageGranularity);
16141	json.WriteString("nonCoherentAtomSize");
16142	json.WriteNumber(deviceProperties.limits.nonCoherentAtomSize);
16143
16144	json.WriteString("memoryHeapCount");
16145	json.WriteNumber(memoryProperties.memoryHeapCount);
16146	json.WriteString("memoryTypeCount");
16147	json.WriteNumber(memoryProperties.memoryTypeCount);
16148	}
16149	json.EndObject();
16150	}
16151	{
16152	json.WriteString("Total");
16153	VmaPrintDetailedStatistics(json, stats.total);
16154	}
16155	{
16156	json.WriteString("MemoryInfo");
16157	json.BeginObject();
16158	{
16159	for (uint32_t heapIndex = `0`; heapIndex < allocator->GetMemoryHeapCount(); ++heapIndex)
16160	{
16161	json.BeginString("Heap ");
16162	json.ContinueString(heapIndex);
16163	json.EndString();
16164	json.BeginObject();
16165	{
16166	const VkMemoryHeap& heapInfo = allocator->m_MemProps.memoryHeaps[heapIndex];
16167	json.WriteString("Flags");
16168	json.BeginArray(true);
16169	{
16170	if (heapInfo.flags & VK_MEMORY_HEAP_DEVICE_LOCAL_BIT)
16171	json.WriteString("DEVICE_LOCAL");
16172	#if VMA_VULKAN_VERSION >= 1001000
16173	if (heapInfo.flags & VK_MEMORY_HEAP_MULTI_INSTANCE_BIT)
16174	json.WriteString("MULTI_INSTANCE");
16175	#endif
16176
16177	VkMemoryHeapFlags flags = heapInfo.flags &
16178	~(VK_MEMORY_HEAP_DEVICE_LOCAL_BIT
16179	#if VMA_VULKAN_VERSION >= 1001000
16180	\| VK_MEMORY_HEAP_MULTI_INSTANCE_BIT
16181	#endif
16182	);
16183	if (flags != `0`)
16184	json.WriteNumber(flags);
16185	}
16186	json.EndArray();
16187
16188	json.WriteString("Size");
16189	json.WriteNumber(heapInfo.size);
16190
16191	json.WriteString("Budget");
16192	json.BeginObject();
16193	{
16194	json.WriteString("BudgetBytes");
16195	json.WriteNumber(budgets[heapIndex].budget);
16196	json.WriteString("UsageBytes");
16197	json.WriteNumber(budgets[heapIndex].usage);
16198	}
16199	json.EndObject();
16200
16201	json.WriteString("Stats");
16202	VmaPrintDetailedStatistics(json, stats.memoryHeap[heapIndex]);
16203
16204	json.WriteString("MemoryPools");
16205	json.BeginObject();
16206	{
16207	for (uint32_t typeIndex = `0`; typeIndex < allocator->GetMemoryTypeCount(); ++typeIndex)
16208	{
16209	if (allocator->MemoryTypeIndexToHeapIndex(typeIndex) == heapIndex)
16210	{
16211	json.BeginString("Type ");
16212	json.ContinueString(typeIndex);
16213	json.EndString();
16214	json.BeginObject();
16215	{
16216	json.WriteString("Flags");
16217	json.BeginArray(true);
16218	{
16219	VkMemoryPropertyFlags flags = allocator->m_MemProps.memoryTypes[typeIndex].propertyFlags;
16220	if (flags & VK_MEMORY_PROPERTY_DEVICE_LOCAL_BIT)
16221	json.WriteString("DEVICE_LOCAL");
16222	if (flags & VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT)
16223	json.WriteString("HOST_VISIBLE");
16224	if (flags & VK_MEMORY_PROPERTY_HOST_COHERENT_BIT)
16225	json.WriteString("HOST_COHERENT");
16226	if (flags & VK_MEMORY_PROPERTY_HOST_CACHED_BIT)
16227	json.WriteString("HOST_CACHED");
16228	if (flags & VK_MEMORY_PROPERTY_LAZILY_ALLOCATED_BIT)
16229	json.WriteString("LAZILY_ALLOCATED");
16230	#if VMA_VULKAN_VERSION >= 1001000
16231	if (flags & VK_MEMORY_PROPERTY_PROTECTED_BIT)
16232	json.WriteString("PROTECTED");
16233	#endif
16234	#if VK_AMD_device_coherent_memory
16235	if (flags & VK_MEMORY_PROPERTY_DEVICE_COHERENT_BIT_AMD_COPY)
16236	json.WriteString("DEVICE_COHERENT_AMD");
16237	if (flags & VK_MEMORY_PROPERTY_DEVICE_UNCACHED_BIT_AMD_COPY)
16238	json.WriteString("DEVICE_UNCACHED_AMD");
16239	#endif
16240
16241	flags &= ~(VK_MEMORY_PROPERTY_DEVICE_LOCAL_BIT
16242	#if VMA_VULKAN_VERSION >= 1001000
16243	\| VK_MEMORY_PROPERTY_LAZILY_ALLOCATED_BIT
16244	#endif
16245	#if VK_AMD_device_coherent_memory
16246	\| VK_MEMORY_PROPERTY_DEVICE_COHERENT_BIT_AMD_COPY
16247	\| VK_MEMORY_PROPERTY_DEVICE_UNCACHED_BIT_AMD_COPY
16248	#endif
16249	\| VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT
16250	\| VK_MEMORY_PROPERTY_HOST_COHERENT_BIT
16251	\| VK_MEMORY_PROPERTY_HOST_CACHED_BIT);
16252	if (flags != `0`)
16253	json.WriteNumber(flags);
16254	}
16255	json.EndArray();
16256
16257	json.WriteString("Stats");
16258	VmaPrintDetailedStatistics(json, stats.memoryType[typeIndex]);
16259	}
16260	json.EndObject();
16261	}
16262	}
16263
16264	}
16265	json.EndObject();
16266	}
16267	json.EndObject();
16268	}
16269	}
16270	json.EndObject();
16271	}
16272
16273	if (detailedMap == VK_TRUE)
16274	allocator->PrintDetailedMap(json);
16275
16276	json.EndObject();
16277	}
16278
16279	*ppStatsString = VmaCreateStringCopy(allocator->GetAllocationCallbacks(), sb.GetData(), sb.GetLength());
16280	}
16281
16282	VMA_CALL_PRE void VMA_CALL_POST vmaFreeStatsString(
16283	VmaAllocator allocator,
16284	char* pStatsString)
16285	{
16286	if(pStatsString != VMA_NULL)
16287	{
16288	VMA_ASSERT(allocator);
16289	VmaFreeString(allocator->GetAllocationCallbacks(), pStatsString);
16290	}
16291	}
16292
16293	#endif // VMA_STATS_STRING_ENABLED
16294
16295	/*
16296	This function is not protected by any mutex because it just reads immutable data.
16297	*/
16298	VMA_CALL_PRE VkResult VMA_CALL_POST vmaFindMemoryTypeIndex(
16299	VmaAllocator allocator,
16300	uint32_t memoryTypeBits,
16301	const VmaAllocationCreateInfo* pAllocationCreateInfo,
16302	uint32_t* pMemoryTypeIndex)
16303	{
16304	VMA_ASSERT(allocator != VK_NULL_HANDLE);
16305	VMA_ASSERT(pAllocationCreateInfo != VMA_NULL);
16306	VMA_ASSERT(pMemoryTypeIndex != VMA_NULL);
16307
16308	return allocator->FindMemoryTypeIndex(memoryTypeBits, pAllocationCreateInfo, UINT32_MAX, pMemoryTypeIndex);
16309	}
16310
16311	VMA_CALL_PRE VkResult VMA_CALL_POST vmaFindMemoryTypeIndexForBufferInfo(
16312	VmaAllocator allocator,
16313	const VkBufferCreateInfo* pBufferCreateInfo,
16314	const VmaAllocationCreateInfo* pAllocationCreateInfo,
16315	uint32_t* pMemoryTypeIndex)
16316	{
16317	VMA_ASSERT(allocator != VK_NULL_HANDLE);
16318	VMA_ASSERT(pBufferCreateInfo != VMA_NULL);
16319	VMA_ASSERT(pAllocationCreateInfo != VMA_NULL);
16320	VMA_ASSERT(pMemoryTypeIndex != VMA_NULL);
16321
16322	const VkDevice hDev = allocator->m_hDevice;
16323	const VmaVulkanFunctions* funcs = &allocator->GetVulkanFunctions();
16324	VkResult res;
16325
16326	#if VMA_VULKAN_VERSION >= 1003000
16327	if(funcs->vkGetDeviceBufferMemoryRequirements)
16328	{
16329	// Can query straight from VkBufferCreateInfo :)
16330	VkDeviceBufferMemoryRequirements devBufMemReq = {VK_STRUCTURE_TYPE_DEVICE_BUFFER_MEMORY_REQUIREMENTS};
16331	devBufMemReq.pCreateInfo = pBufferCreateInfo;
16332
16333	VkMemoryRequirements2 memReq = {VK_STRUCTURE_TYPE_MEMORY_REQUIREMENTS_2};
16334	(*funcs->vkGetDeviceBufferMemoryRequirements)(hDev, &devBufMemReq, &memReq);
16335
16336	res = allocator->FindMemoryTypeIndex(
16337	memReq.memoryRequirements.memoryTypeBits, pAllocationCreateInfo, pBufferCreateInfo->usage, pMemoryTypeIndex);
16338	}
16339	else
16340	#endif // #if VMA_VULKAN_VERSION >= 1003000
16341	{
16342	// Must create a dummy buffer to query :(
16343	VkBuffer hBuffer = VK_NULL_HANDLE;
16344	res = funcs->vkCreateBuffer(
16345	hDev, pBufferCreateInfo, allocator->GetAllocationCallbacks(), &hBuffer);
16346	if(res == VK_SUCCESS)
16347	{
16348	VkMemoryRequirements memReq = {};
16349	funcs->vkGetBufferMemoryRequirements(hDev, hBuffer, &memReq);
16350
16351	res = allocator->FindMemoryTypeIndex(
16352	memReq.memoryTypeBits, pAllocationCreateInfo, pBufferCreateInfo->usage, pMemoryTypeIndex);
16353
16354	funcs->vkDestroyBuffer(
16355	hDev, hBuffer, allocator->GetAllocationCallbacks());
16356	}
16357	}
16358	return res;
16359	}
16360
16361	VMA_CALL_PRE VkResult VMA_CALL_POST vmaFindMemoryTypeIndexForImageInfo(
16362	VmaAllocator allocator,
16363	const VkImageCreateInfo* pImageCreateInfo,
16364	const VmaAllocationCreateInfo* pAllocationCreateInfo,
16365	uint32_t* pMemoryTypeIndex)
16366	{
16367	VMA_ASSERT(allocator != VK_NULL_HANDLE);
16368	VMA_ASSERT(pImageCreateInfo != VMA_NULL);
16369	VMA_ASSERT(pAllocationCreateInfo != VMA_NULL);
16370	VMA_ASSERT(pMemoryTypeIndex != VMA_NULL);
16371
16372	const VkDevice hDev = allocator->m_hDevice;
16373	const VmaVulkanFunctions* funcs = &allocator->GetVulkanFunctions();
16374	VkResult res;
16375
16376	#if VMA_VULKAN_VERSION >= 1003000
16377	if(funcs->vkGetDeviceImageMemoryRequirements)
16378	{
16379	// Can query straight from VkImageCreateInfo :)
16380	VkDeviceImageMemoryRequirements devImgMemReq = {VK_STRUCTURE_TYPE_DEVICE_IMAGE_MEMORY_REQUIREMENTS};
16381	devImgMemReq.pCreateInfo = pImageCreateInfo;
16382	VMA_ASSERT(pImageCreateInfo->tiling != VK_IMAGE_TILING_DRM_FORMAT_MODIFIER_EXT_COPY && (pImageCreateInfo->flags & VK_IMAGE_CREATE_DISJOINT_BIT_COPY) == `0` &&
16383	"Cannot use this VkImageCreateInfo with vmaFindMemoryTypeIndexForImageInfo as I don't know what to pass as VkDeviceImageMemoryRequirements::planeAspect.");
16384
16385	VkMemoryRequirements2 memReq = {VK_STRUCTURE_TYPE_MEMORY_REQUIREMENTS_2};
16386	(*funcs->vkGetDeviceImageMemoryRequirements)(hDev, &devImgMemReq, &memReq);
16387
16388	res = allocator->FindMemoryTypeIndex(
16389	memReq.memoryRequirements.memoryTypeBits, pAllocationCreateInfo, pImageCreateInfo->usage, pMemoryTypeIndex);
16390	}
16391	else
16392	#endif // #if VMA_VULKAN_VERSION >= 1003000
16393	{
16394	// Must create a dummy image to query :(
16395	VkImage hImage = VK_NULL_HANDLE;
16396	res = funcs->vkCreateImage(
16397	hDev, pImageCreateInfo, allocator->GetAllocationCallbacks(), &hImage);
16398	if(res == VK_SUCCESS)
16399	{
16400	VkMemoryRequirements memReq = {};
16401	funcs->vkGetImageMemoryRequirements(hDev, hImage, &memReq);
16402
16403	res = allocator->FindMemoryTypeIndex(
16404	memReq.memoryTypeBits, pAllocationCreateInfo, pImageCreateInfo->usage, pMemoryTypeIndex);
16405
16406	funcs->vkDestroyImage(
16407	hDev, hImage, allocator->GetAllocationCallbacks());
16408	}
16409	}
16410	return res;
16411	}
16412
16413	VMA_CALL_PRE VkResult VMA_CALL_POST vmaCreatePool(
16414	VmaAllocator allocator,
16415	const VmaPoolCreateInfo* pCreateInfo,
16416	VmaPool* pPool)
16417	{
16418	VMA_ASSERT(allocator && pCreateInfo && pPool);
16419
16420	VMA_DEBUG_LOG("vmaCreatePool");
16421
16422	VMA_DEBUG_GLOBAL_MUTEX_LOCK
16423
16424	return allocator->CreatePool(pCreateInfo, pPool);
16425	}
16426
16427	VMA_CALL_PRE void VMA_CALL_POST vmaDestroyPool(
16428	VmaAllocator allocator,
16429	VmaPool pool)
16430	{
16431	VMA_ASSERT(allocator);
16432
16433	if(pool == VK_NULL_HANDLE)
16434	{
16435	return;
16436	}
16437
16438	VMA_DEBUG_LOG("vmaDestroyPool");
16439
16440	VMA_DEBUG_GLOBAL_MUTEX_LOCK
16441
16442	allocator->DestroyPool(pool);
16443	}
16444
16445	VMA_CALL_PRE void VMA_CALL_POST vmaGetPoolStatistics(
16446	VmaAllocator allocator,
16447	VmaPool pool,
16448	VmaStatistics* pPoolStats)
16449	{
16450	VMA_ASSERT(allocator && pool && pPoolStats);
16451
16452	VMA_DEBUG_GLOBAL_MUTEX_LOCK
16453
16454	allocator->GetPoolStatistics(pool, pPoolStats);
16455	}
16456
16457	VMA_CALL_PRE void VMA_CALL_POST vmaCalculatePoolStatistics(
16458	VmaAllocator allocator,
16459	VmaPool pool,
16460	VmaDetailedStatistics* pPoolStats)
16461	{
16462	VMA_ASSERT(allocator && pool && pPoolStats);
16463
16464	VMA_DEBUG_GLOBAL_MUTEX_LOCK
16465
16466	allocator->CalculatePoolStatistics(pool, pPoolStats);
16467	}
16468
16469	VMA_CALL_PRE VkResult VMA_CALL_POST vmaCheckPoolCorruption(VmaAllocator allocator, VmaPool pool)
16470	{
16471	VMA_ASSERT(allocator && pool);
16472
16473	VMA_DEBUG_GLOBAL_MUTEX_LOCK
16474
16475	VMA_DEBUG_LOG("vmaCheckPoolCorruption");
16476
16477	return allocator->CheckPoolCorruption(pool);
16478	}
16479
16480	VMA_CALL_PRE void VMA_CALL_POST vmaGetPoolName(
16481	VmaAllocator allocator,
16482	VmaPool pool,
16483	const char** ppName)
16484	{
16485	VMA_ASSERT(allocator && pool && ppName);
16486
16487	VMA_DEBUG_LOG("vmaGetPoolName");
16488
16489	VMA_DEBUG_GLOBAL_MUTEX_LOCK
16490
16491	*ppName = pool->GetName();
16492	}
16493
16494	VMA_CALL_PRE void VMA_CALL_POST vmaSetPoolName(
16495	VmaAllocator allocator,
16496	VmaPool pool,
16497	const char* pName)
16498	{
16499	VMA_ASSERT(allocator && pool);
16500
16501	VMA_DEBUG_LOG("vmaSetPoolName");
16502
16503	VMA_DEBUG_GLOBAL_MUTEX_LOCK
16504
16505	pool->SetName(pName);
16506	}
16507
16508	VMA_CALL_PRE VkResult VMA_CALL_POST vmaAllocateMemory(
16509	VmaAllocator allocator,
16510	const VkMemoryRequirements* pVkMemoryRequirements,
16511	const VmaAllocationCreateInfo* pCreateInfo,
16512	VmaAllocation* pAllocation,
16513	VmaAllocationInfo* pAllocationInfo)
16514	{
16515	VMA_ASSERT(allocator && pVkMemoryRequirements && pCreateInfo && pAllocation);
16516
16517	VMA_DEBUG_LOG("vmaAllocateMemory");
16518
16519	VMA_DEBUG_GLOBAL_MUTEX_LOCK
16520
16521	VkResult result = allocator->AllocateMemory(
16522	*pVkMemoryRequirements,
16523	false, // requiresDedicatedAllocation
16524	false, // prefersDedicatedAllocation
16525	VK_NULL_HANDLE, // dedicatedBuffer
16526	VK_NULL_HANDLE, // dedicatedImage
16527	UINT32_MAX, // dedicatedBufferImageUsage
16528	*pCreateInfo,
16529	VMA_SUBALLOCATION_TYPE_UNKNOWN,
16530	`1`, // allocationCount
16531	pAllocation);
16532
16533	if(pAllocationInfo != VMA_NULL && result == VK_SUCCESS)
16534	{
16535	allocator->GetAllocationInfo(*pAllocation, pAllocationInfo);
16536	}
16537
16538	return result;
16539	}
16540
16541	VMA_CALL_PRE VkResult VMA_CALL_POST vmaAllocateMemoryPages(
16542	VmaAllocator allocator,
16543	const VkMemoryRequirements* pVkMemoryRequirements,
16544	const VmaAllocationCreateInfo* pCreateInfo,
16545	size_t allocationCount,
16546	VmaAllocation* pAllocations,
16547	VmaAllocationInfo* pAllocationInfo)
16548	{
16549	if(allocationCount == `0`)
16550	{
16551	return VK_SUCCESS;
16552	}
16553
16554	VMA_ASSERT(allocator && pVkMemoryRequirements && pCreateInfo && pAllocations);
16555
16556	VMA_DEBUG_LOG("vmaAllocateMemoryPages");
16557
16558	VMA_DEBUG_GLOBAL_MUTEX_LOCK
16559
16560	VkResult result = allocator->AllocateMemory(
16561	*pVkMemoryRequirements,
16562	false, // requiresDedicatedAllocation
16563	false, // prefersDedicatedAllocation
16564	VK_NULL_HANDLE, // dedicatedBuffer
16565	VK_NULL_HANDLE, // dedicatedImage
16566	UINT32_MAX, // dedicatedBufferImageUsage
16567	*pCreateInfo,
16568	VMA_SUBALLOCATION_TYPE_UNKNOWN,
16569	allocationCount,
16570	pAllocations);
16571
16572	if(pAllocationInfo != VMA_NULL && result == VK_SUCCESS)
16573	{
16574	for(size_t i = `0`; i < allocationCount; ++i)
16575	{
16576	allocator->GetAllocationInfo(pAllocations[i], pAllocationInfo + i);
16577	}
16578	}
16579
16580	return result;
16581	}
16582
16583	VMA_CALL_PRE VkResult VMA_CALL_POST vmaAllocateMemoryForBuffer(
16584	VmaAllocator allocator,
16585	VkBuffer buffer,
16586	const VmaAllocationCreateInfo* pCreateInfo,
16587	VmaAllocation* pAllocation,
16588	VmaAllocationInfo* pAllocationInfo)
16589	{
16590	VMA_ASSERT(allocator && buffer != VK_NULL_HANDLE && pCreateInfo && pAllocation);
16591
16592	VMA_DEBUG_LOG("vmaAllocateMemoryForBuffer");
16593
16594	VMA_DEBUG_GLOBAL_MUTEX_LOCK
16595
16596	VkMemoryRequirements vkMemReq = {};
16597	bool requiresDedicatedAllocation = false;
16598	bool prefersDedicatedAllocation = false;
16599	allocator->GetBufferMemoryRequirements(buffer, vkMemReq,
16600	requiresDedicatedAllocation,
16601	prefersDedicatedAllocation);
16602
16603	VkResult result = allocator->AllocateMemory(
16604	vkMemReq,
16605	requiresDedicatedAllocation,
16606	prefersDedicatedAllocation,
16607	buffer, // dedicatedBuffer
16608	VK_NULL_HANDLE, // dedicatedImage
16609	UINT32_MAX, // dedicatedBufferImageUsage
16610	*pCreateInfo,
16611	VMA_SUBALLOCATION_TYPE_BUFFER,
16612	`1`, // allocationCount
16613	pAllocation);
16614
16615	if(pAllocationInfo && result == VK_SUCCESS)
16616	{
16617	allocator->GetAllocationInfo(*pAllocation, pAllocationInfo);
16618	}
16619
16620	return result;
16621	}
16622
16623	VMA_CALL_PRE VkResult VMA_CALL_POST vmaAllocateMemoryForImage(
16624	VmaAllocator allocator,
16625	VkImage image,
16626	const VmaAllocationCreateInfo* pCreateInfo,
16627	VmaAllocation* pAllocation,
16628	VmaAllocationInfo* pAllocationInfo)
16629	{
16630	VMA_ASSERT(allocator && image != VK_NULL_HANDLE && pCreateInfo && pAllocation);
16631
16632	VMA_DEBUG_LOG("vmaAllocateMemoryForImage");
16633
16634	VMA_DEBUG_GLOBAL_MUTEX_LOCK
16635
16636	VkMemoryRequirements vkMemReq = {};
16637	bool requiresDedicatedAllocation = false;
16638	bool prefersDedicatedAllocation = false;
16639	allocator->GetImageMemoryRequirements(image, vkMemReq,
16640	requiresDedicatedAllocation, prefersDedicatedAllocation);
16641
16642	VkResult result = allocator->AllocateMemory(
16643	vkMemReq,
16644	requiresDedicatedAllocation,
16645	prefersDedicatedAllocation,
16646	VK_NULL_HANDLE, // dedicatedBuffer
16647	image, // dedicatedImage
16648	UINT32_MAX, // dedicatedBufferImageUsage
16649	*pCreateInfo,
16650	VMA_SUBALLOCATION_TYPE_IMAGE_UNKNOWN,
16651	`1`, // allocationCount
16652	pAllocation);
16653
16654	if(pAllocationInfo && result == VK_SUCCESS)
16655	{
16656	allocator->GetAllocationInfo(*pAllocation, pAllocationInfo);
16657	}
16658
16659	return result;
16660	}
16661
16662	VMA_CALL_PRE void VMA_CALL_POST vmaFreeMemory(
16663	VmaAllocator allocator,
16664	VmaAllocation allocation)
16665	{
16666	VMA_ASSERT(allocator);
16667
16668	if(allocation == VK_NULL_HANDLE)
16669	{
16670	return;
16671	}
16672
16673	VMA_DEBUG_LOG("vmaFreeMemory");
16674
16675	VMA_DEBUG_GLOBAL_MUTEX_LOCK
16676
16677	allocator->FreeMemory(
16678	`1`, // allocationCount
16679	&allocation);
16680	}
16681
16682	VMA_CALL_PRE void VMA_CALL_POST vmaFreeMemoryPages(
16683	VmaAllocator allocator,
16684	size_t allocationCount,
16685	const VmaAllocation* pAllocations)
16686	{
16687	if(allocationCount == `0`)
16688	{
16689	return;
16690	}
16691
16692	VMA_ASSERT(allocator);
16693
16694	VMA_DEBUG_LOG("vmaFreeMemoryPages");
16695
16696	VMA_DEBUG_GLOBAL_MUTEX_LOCK
16697
16698	allocator->FreeMemory(allocationCount, pAllocations);
16699	}
16700
16701	VMA_CALL_PRE void VMA_CALL_POST vmaGetAllocationInfo(
16702	VmaAllocator allocator,
16703	VmaAllocation allocation,
16704	VmaAllocationInfo* pAllocationInfo)
16705	{
16706	VMA_ASSERT(allocator && allocation && pAllocationInfo);
16707
16708	VMA_DEBUG_GLOBAL_MUTEX_LOCK
16709
16710	allocator->GetAllocationInfo(allocation, pAllocationInfo);
16711	}
16712
16713	VMA_CALL_PRE void VMA_CALL_POST vmaSetAllocationUserData(
16714	VmaAllocator allocator,
16715	VmaAllocation allocation,
16716	void* pUserData)
16717	{
16718	VMA_ASSERT(allocator && allocation);
16719
16720	VMA_DEBUG_GLOBAL_MUTEX_LOCK
16721
16722	allocation->SetUserData(allocator, pUserData);
16723	}
16724
16725	VMA_CALL_PRE void VMA_CALL_POST vmaSetAllocationName(
16726	VmaAllocator VMA_NOT_NULL allocator,
16727	VmaAllocation VMA_NOT_NULL allocation,
16728	const char* VMA_NULLABLE pName)
16729	{
16730	allocation->SetName(allocator, pName);
16731	}
16732
16733	VMA_CALL_PRE void VMA_CALL_POST vmaGetAllocationMemoryProperties(
16734	VmaAllocator VMA_NOT_NULL allocator,
16735	VmaAllocation VMA_NOT_NULL allocation,
16736	VkMemoryPropertyFlags* VMA_NOT_NULL pFlags)
16737	{
16738	VMA_ASSERT(allocator && allocation && pFlags);
16739	const uint32_t memTypeIndex = allocation->GetMemoryTypeIndex();
16740	*pFlags = allocator->m_MemProps.memoryTypes[memTypeIndex].propertyFlags;
16741	}
16742
16743	VMA_CALL_PRE VkResult VMA_CALL_POST vmaMapMemory(
16744	VmaAllocator allocator,
16745	VmaAllocation allocation,
16746	void** ppData)
16747	{
16748	VMA_ASSERT(allocator && allocation && ppData);
16749
16750	VMA_DEBUG_GLOBAL_MUTEX_LOCK
16751
16752	return allocator->Map(allocation, ppData);
16753	}
16754
16755	VMA_CALL_PRE void VMA_CALL_POST vmaUnmapMemory(
16756	VmaAllocator allocator,
16757	VmaAllocation allocation)
16758	{
16759	VMA_ASSERT(allocator && allocation);
16760
16761	VMA_DEBUG_GLOBAL_MUTEX_LOCK
16762
16763	allocator->Unmap(allocation);
16764	}
16765
16766	VMA_CALL_PRE VkResult VMA_CALL_POST vmaFlushAllocation(
16767	VmaAllocator allocator,
16768	VmaAllocation allocation,
16769	VkDeviceSize offset,
16770	VkDeviceSize size)
16771	{
16772	VMA_ASSERT(allocator && allocation);
16773
16774	VMA_DEBUG_LOG("vmaFlushAllocation");
16775
16776	VMA_DEBUG_GLOBAL_MUTEX_LOCK
16777
16778	const VkResult res = allocator->FlushOrInvalidateAllocation(allocation, offset, size, VMA_CACHE_FLUSH);
16779
16780	return res;
16781	}
16782
16783	VMA_CALL_PRE VkResult VMA_CALL_POST vmaInvalidateAllocation(
16784	VmaAllocator allocator,
16785	VmaAllocation allocation,
16786	VkDeviceSize offset,
16787	VkDeviceSize size)
16788	{
16789	VMA_ASSERT(allocator && allocation);
16790
16791	VMA_DEBUG_LOG("vmaInvalidateAllocation");
16792
16793	VMA_DEBUG_GLOBAL_MUTEX_LOCK
16794
16795	const VkResult res = allocator->FlushOrInvalidateAllocation(allocation, offset, size, VMA_CACHE_INVALIDATE);
16796
16797	return res;
16798	}
16799
16800	VMA_CALL_PRE VkResult VMA_CALL_POST vmaFlushAllocations(
16801	VmaAllocator allocator,
16802	uint32_t allocationCount,
16803	const VmaAllocation* allocations,
16804	const VkDeviceSize* offsets,
16805	const VkDeviceSize* sizes)
16806	{
16807	VMA_ASSERT(allocator);
16808
16809	if(allocationCount == `0`)
16810	{
16811	return VK_SUCCESS;
16812	}
16813
16814	VMA_ASSERT(allocations);
16815
16816	VMA_DEBUG_LOG("vmaFlushAllocations");
16817
16818	VMA_DEBUG_GLOBAL_MUTEX_LOCK
16819
16820	const VkResult res = allocator->FlushOrInvalidateAllocations(allocationCount, allocations, offsets, sizes, VMA_CACHE_FLUSH);
16821
16822	return res;
16823	}
16824
16825	VMA_CALL_PRE VkResult VMA_CALL_POST vmaInvalidateAllocations(
16826	VmaAllocator allocator,
16827	uint32_t allocationCount,
16828	const VmaAllocation* allocations,
16829	const VkDeviceSize* offsets,
16830	const VkDeviceSize* sizes)
16831	{
16832	VMA_ASSERT(allocator);
16833
16834	if(allocationCount == `0`)
16835	{
16836	return VK_SUCCESS;
16837	}
16838
16839	VMA_ASSERT(allocations);
16840
16841	VMA_DEBUG_LOG("vmaInvalidateAllocations");
16842
16843	VMA_DEBUG_GLOBAL_MUTEX_LOCK
16844
16845	const VkResult res = allocator->FlushOrInvalidateAllocations(allocationCount, allocations, offsets, sizes, VMA_CACHE_INVALIDATE);
16846
16847	return res;
16848	}
16849
16850	VMA_CALL_PRE VkResult VMA_CALL_POST vmaCheckCorruption(
16851	VmaAllocator allocator,
16852	uint32_t memoryTypeBits)
16853	{
16854	VMA_ASSERT(allocator);
16855
16856	VMA_DEBUG_LOG("vmaCheckCorruption");
16857
16858	VMA_DEBUG_GLOBAL_MUTEX_LOCK
16859
16860	return allocator->CheckCorruption(memoryTypeBits);
16861	}
16862
16863	VMA_CALL_PRE VkResult VMA_CALL_POST vmaBeginDefragmentation(
16864	VmaAllocator allocator,
16865	const VmaDefragmentationInfo* pInfo,
16866	VmaDefragmentationContext* pContext)
16867	{
16868	VMA_ASSERT(allocator && pInfo && pContext);
16869
16870	VMA_DEBUG_LOG("vmaBeginDefragmentation");
16871
16872	if (pInfo->pool != VMA_NULL)
16873	{
16874	// Check if run on supported algorithms
16875	if (pInfo->pool->m_BlockVector.GetAlgorithm() & VMA_POOL_CREATE_LINEAR_ALGORITHM_BIT)
16876	return VK_ERROR_FEATURE_NOT_PRESENT;
16877	}
16878
16879	VMA_DEBUG_GLOBAL_MUTEX_LOCK
16880
16881	pContext = vma_new(allocator, VmaDefragmentationContext_T)(allocator, pInfo);
16882	return VK_SUCCESS;
16883	}
16884
16885	VMA_CALL_PRE void VMA_CALL_POST vmaEndDefragmentation(
16886	VmaAllocator allocator,
16887	VmaDefragmentationContext context,
16888	VmaDefragmentationStats* pStats)
16889	{
16890	VMA_ASSERT(allocator && context);
16891
16892	VMA_DEBUG_LOG("vmaEndDefragmentation");
16893
16894	VMA_DEBUG_GLOBAL_MUTEX_LOCK
16895
16896	if (pStats)
16897	context->GetStats(*pStats);
16898	vma_delete(allocator, context);
16899	}
16900
16901	VMA_CALL_PRE VkResult VMA_CALL_POST vmaBeginDefragmentationPass(
16902	VmaAllocator VMA_NOT_NULL allocator,
16903	VmaDefragmentationContext VMA_NOT_NULL context,
16904	VmaDefragmentationPassMoveInfo* VMA_NOT_NULL pPassInfo)
16905	{
16906	VMA_ASSERT(context && pPassInfo);
16907
16908	VMA_DEBUG_LOG("vmaBeginDefragmentationPass");
16909
16910	VMA_DEBUG_GLOBAL_MUTEX_LOCK
16911
16912	return context->DefragmentPassBegin(*pPassInfo);
16913	}
16914
16915	VMA_CALL_PRE VkResult VMA_CALL_POST vmaEndDefragmentationPass(
16916	VmaAllocator VMA_NOT_NULL allocator,
16917	VmaDefragmentationContext VMA_NOT_NULL context,
16918	VmaDefragmentationPassMoveInfo* VMA_NOT_NULL pPassInfo)
16919	{
16920	VMA_ASSERT(context && pPassInfo);
16921
16922	VMA_DEBUG_LOG("vmaEndDefragmentationPass");
16923
16924	VMA_DEBUG_GLOBAL_MUTEX_LOCK
16925
16926	return context->DefragmentPassEnd(*pPassInfo);
16927	}
16928
16929	VMA_CALL_PRE VkResult VMA_CALL_POST vmaBindBufferMemory(
16930	VmaAllocator allocator,
16931	VmaAllocation allocation,
16932	VkBuffer buffer)
16933	{
16934	VMA_ASSERT(allocator && allocation && buffer);
16935
16936	VMA_DEBUG_LOG("vmaBindBufferMemory");
16937
16938	VMA_DEBUG_GLOBAL_MUTEX_LOCK
16939
16940	return allocator->BindBufferMemory(allocation, `0`, buffer, VMA_NULL);
16941	}
16942
16943	VMA_CALL_PRE VkResult VMA_CALL_POST vmaBindBufferMemory2(
16944	VmaAllocator allocator,
16945	VmaAllocation allocation,
16946	VkDeviceSize allocationLocalOffset,
16947	VkBuffer buffer,
16948	const void* pNext)
16949	{
16950	VMA_ASSERT(allocator && allocation && buffer);
16951
16952	VMA_DEBUG_LOG("vmaBindBufferMemory2");
16953
16954	VMA_DEBUG_GLOBAL_MUTEX_LOCK
16955
16956	return allocator->BindBufferMemory(allocation, allocationLocalOffset, buffer, pNext);
16957	}
16958
16959	VMA_CALL_PRE VkResult VMA_CALL_POST vmaBindImageMemory(
16960	VmaAllocator allocator,
16961	VmaAllocation allocation,
16962	VkImage image)
16963	{
16964	VMA_ASSERT(allocator && allocation && image);
16965
16966	VMA_DEBUG_LOG("vmaBindImageMemory");
16967
16968	VMA_DEBUG_GLOBAL_MUTEX_LOCK
16969
16970	return allocator->BindImageMemory(allocation, `0`, image, VMA_NULL);
16971	}
16972
16973	VMA_CALL_PRE VkResult VMA_CALL_POST vmaBindImageMemory2(
16974	VmaAllocator allocator,
16975	VmaAllocation allocation,
16976	VkDeviceSize allocationLocalOffset,
16977	VkImage image,
16978	const void* pNext)
16979	{
16980	VMA_ASSERT(allocator && allocation && image);
16981
16982	VMA_DEBUG_LOG("vmaBindImageMemory2");
16983
16984	VMA_DEBUG_GLOBAL_MUTEX_LOCK
16985
16986	return allocator->BindImageMemory(allocation, allocationLocalOffset, image, pNext);
16987	}
16988
16989	VMA_CALL_PRE VkResult VMA_CALL_POST vmaCreateBuffer(
16990	VmaAllocator allocator,
16991	const VkBufferCreateInfo* pBufferCreateInfo,
16992	const VmaAllocationCreateInfo* pAllocationCreateInfo,
16993	VkBuffer* pBuffer,
16994	VmaAllocation* pAllocation,
16995	VmaAllocationInfo* pAllocationInfo)
16996	{
16997	VMA_ASSERT(allocator && pBufferCreateInfo && pAllocationCreateInfo && pBuffer && pAllocation);
16998
16999	if(pBufferCreateInfo->size == `0`)
17000	{
17001	return VK_ERROR_INITIALIZATION_FAILED;
17002	}
17003	if((pBufferCreateInfo->usage & VK_BUFFER_USAGE_SHADER_DEVICE_ADDRESS_BIT_COPY) != `0` &&
17004	!allocator->m_UseKhrBufferDeviceAddress)
17005	{
17006	VMA_ASSERT(`0` && "Creating a buffer with VK_BUFFER_USAGE_SHADER_DEVICE_ADDRESS_BIT is not valid if VMA_ALLOCATOR_CREATE_BUFFER_DEVICE_ADDRESS_BIT was not used.");
17007	return VK_ERROR_INITIALIZATION_FAILED;
17008	}
17009
17010	VMA_DEBUG_LOG("vmaCreateBuffer");
17011
17012	VMA_DEBUG_GLOBAL_MUTEX_LOCK
17013
17014	*pBuffer = VK_NULL_HANDLE;
17015	*pAllocation = VK_NULL_HANDLE;
17016
17017	// 1. Create VkBuffer.
17018	VkResult res = (*allocator->GetVulkanFunctions().vkCreateBuffer)(
17019	allocator->m_hDevice,
17020	pBufferCreateInfo,
17021	allocator->GetAllocationCallbacks(),
17022	pBuffer);
17023	if(res >= `0`)
17024	{
17025	// 2. vkGetBufferMemoryRequirements.
17026	VkMemoryRequirements vkMemReq = {};
17027	bool requiresDedicatedAllocation = false;
17028	bool prefersDedicatedAllocation = false;
17029	allocator->GetBufferMemoryRequirements(*pBuffer, vkMemReq,
17030	requiresDedicatedAllocation, prefersDedicatedAllocation);
17031
17032	// 3. Allocate memory using allocator.
17033	res = allocator->AllocateMemory(
17034	vkMemReq,
17035	requiresDedicatedAllocation,
17036	prefersDedicatedAllocation,
17037	pBuffer, // dedicatedBuffer*
17038	VK_NULL_HANDLE, // dedicatedImage
17039	pBufferCreateInfo->usage, // dedicatedBufferImageUsage
17040	*pAllocationCreateInfo,
17041	VMA_SUBALLOCATION_TYPE_BUFFER,
17042	`1`, // allocationCount
17043	pAllocation);
17044
17045	if(res >= `0`)
17046	{
17047	// 3. Bind buffer with memory.
17048	if((pAllocationCreateInfo->flags & VMA_ALLOCATION_CREATE_DONT_BIND_BIT) == `0`)
17049	{
17050	res = allocator->BindBufferMemory(pAllocation, `0`, pBuffer, VMA_NULL);
17051	}
17052	if(res >= `0`)
17053	{
17054	// All steps succeeded.
17055	#if VMA_STATS_STRING_ENABLED
17056	(*pAllocation)->InitBufferImageUsage(pBufferCreateInfo->usage);
17057	#endif
17058	if(pAllocationInfo != VMA_NULL)
17059	{
17060	allocator->GetAllocationInfo(*pAllocation, pAllocationInfo);
17061	}
17062
17063	return VK_SUCCESS;
17064	}
17065	allocator->FreeMemory(
17066	`1`, // allocationCount
17067	pAllocation);
17068	*pAllocation = VK_NULL_HANDLE;
17069	(allocator->GetVulkanFunctions().vkDestroyBuffer)(allocator->m_hDevice, pBuffer, allocator->GetAllocationCallbacks());
17070	*pBuffer = VK_NULL_HANDLE;
17071	return res;
17072	}
17073	(allocator->GetVulkanFunctions().vkDestroyBuffer)(allocator->m_hDevice, pBuffer, allocator->GetAllocationCallbacks());
17074	*pBuffer = VK_NULL_HANDLE;
17075	return res;
17076	}
17077	return res;
17078	}
17079
17080	VMA_CALL_PRE VkResult VMA_CALL_POST vmaCreateBufferWithAlignment(
17081	VmaAllocator allocator,
17082	const VkBufferCreateInfo* pBufferCreateInfo,
17083	const VmaAllocationCreateInfo* pAllocationCreateInfo,
17084	VkDeviceSize minAlignment,
17085	VkBuffer* pBuffer,
17086	VmaAllocation* pAllocation,
17087	VmaAllocationInfo* pAllocationInfo)
17088	{
17089	VMA_ASSERT(allocator && pBufferCreateInfo && pAllocationCreateInfo && VmaIsPow2(minAlignment) && pBuffer && pAllocation);
17090
17091	if(pBufferCreateInfo->size == `0`)
17092	{
17093	return VK_ERROR_INITIALIZATION_FAILED;
17094	}
17095	if((pBufferCreateInfo->usage & VK_BUFFER_USAGE_SHADER_DEVICE_ADDRESS_BIT_COPY) != `0` &&
17096	!allocator->m_UseKhrBufferDeviceAddress)
17097	{
17098	VMA_ASSERT(`0` && "Creating a buffer with VK_BUFFER_USAGE_SHADER_DEVICE_ADDRESS_BIT is not valid if VMA_ALLOCATOR_CREATE_BUFFER_DEVICE_ADDRESS_BIT was not used.");
17099	return VK_ERROR_INITIALIZATION_FAILED;
17100	}
17101
17102	VMA_DEBUG_LOG("vmaCreateBufferWithAlignment");
17103
17104	VMA_DEBUG_GLOBAL_MUTEX_LOCK
17105
17106	*pBuffer = VK_NULL_HANDLE;
17107	*pAllocation = VK_NULL_HANDLE;
17108
17109	// 1. Create VkBuffer.
17110	VkResult res = (*allocator->GetVulkanFunctions().vkCreateBuffer)(
17111	allocator->m_hDevice,
17112	pBufferCreateInfo,
17113	allocator->GetAllocationCallbacks(),
17114	pBuffer);
17115	if(res >= `0`)
17116	{
17117	// 2. vkGetBufferMemoryRequirements.
17118	VkMemoryRequirements vkMemReq = {};
17119	bool requiresDedicatedAllocation = false;
17120	bool prefersDedicatedAllocation = false;
17121	allocator->GetBufferMemoryRequirements(*pBuffer, vkMemReq,
17122	requiresDedicatedAllocation, prefersDedicatedAllocation);
17123
17124	// 2a. Include minAlignment
17125	vkMemReq.alignment = VMA_MAX(vkMemReq.alignment, minAlignment);
17126
17127	// 3. Allocate memory using allocator.
17128	res = allocator->AllocateMemory(
17129	vkMemReq,
17130	requiresDedicatedAllocation,
17131	prefersDedicatedAllocation,
17132	pBuffer, // dedicatedBuffer*
17133	VK_NULL_HANDLE, // dedicatedImage
17134	pBufferCreateInfo->usage, // dedicatedBufferImageUsage
17135	*pAllocationCreateInfo,
17136	VMA_SUBALLOCATION_TYPE_BUFFER,
17137	`1`, // allocationCount
17138	pAllocation);
17139
17140	if(res >= `0`)
17141	{
17142	// 3. Bind buffer with memory.
17143	if((pAllocationCreateInfo->flags & VMA_ALLOCATION_CREATE_DONT_BIND_BIT) == `0`)
17144	{
17145	res = allocator->BindBufferMemory(pAllocation, `0`, pBuffer, VMA_NULL);
17146	}
17147	if(res >= `0`)
17148	{
17149	// All steps succeeded.
17150	#if VMA_STATS_STRING_ENABLED
17151	(*pAllocation)->InitBufferImageUsage(pBufferCreateInfo->usage);
17152	#endif
17153	if(pAllocationInfo != VMA_NULL)
17154	{
17155	allocator->GetAllocationInfo(*pAllocation, pAllocationInfo);
17156	}
17157
17158	return VK_SUCCESS;
17159	}
17160	allocator->FreeMemory(
17161	`1`, // allocationCount
17162	pAllocation);
17163	*pAllocation = VK_NULL_HANDLE;
17164	(allocator->GetVulkanFunctions().vkDestroyBuffer)(allocator->m_hDevice, pBuffer, allocator->GetAllocationCallbacks());
17165	*pBuffer = VK_NULL_HANDLE;
17166	return res;
17167	}
17168	(allocator->GetVulkanFunctions().vkDestroyBuffer)(allocator->m_hDevice, pBuffer, allocator->GetAllocationCallbacks());
17169	*pBuffer = VK_NULL_HANDLE;
17170	return res;
17171	}
17172	return res;
17173	}
17174
17175	VMA_CALL_PRE VkResult VMA_CALL_POST vmaCreateAliasingBuffer(
17176	VmaAllocator VMA_NOT_NULL allocator,
17177	VmaAllocation VMA_NOT_NULL allocation,
17178	const VkBufferCreateInfo* VMA_NOT_NULL pBufferCreateInfo,
17179	VkBuffer VMA_NULLABLE_NON_DISPATCHABLE* VMA_NOT_NULL pBuffer)
17180	{
17181	VMA_ASSERT(allocator && pBufferCreateInfo && pBuffer && allocation);
17182
17183	VMA_DEBUG_LOG("vmaCreateAliasingBuffer");
17184
17185	*pBuffer = VK_NULL_HANDLE;
17186
17187	if (pBufferCreateInfo->size == `0`)
17188	{
17189	return VK_ERROR_INITIALIZATION_FAILED;
17190	}
17191	if ((pBufferCreateInfo->usage & VK_BUFFER_USAGE_SHADER_DEVICE_ADDRESS_BIT_COPY) != `0` &&
17192	!allocator->m_UseKhrBufferDeviceAddress)
17193	{
17194	VMA_ASSERT(`0` && "Creating a buffer with VK_BUFFER_USAGE_SHADER_DEVICE_ADDRESS_BIT is not valid if VMA_ALLOCATOR_CREATE_BUFFER_DEVICE_ADDRESS_BIT was not used.");
17195	return VK_ERROR_INITIALIZATION_FAILED;
17196	}
17197
17198	VMA_DEBUG_GLOBAL_MUTEX_LOCK
17199
17200	// 1. Create VkBuffer.
17201	VkResult res = (*allocator->GetVulkanFunctions().vkCreateBuffer)(
17202	allocator->m_hDevice,
17203	pBufferCreateInfo,
17204	allocator->GetAllocationCallbacks(),
17205	pBuffer);
17206	if (res >= `0`)
17207	{
17208	// 2. Bind buffer with memory.
17209	res = allocator->BindBufferMemory(allocation, `0`, *pBuffer, VMA_NULL);
17210	if (res >= `0`)
17211	{
17212	return VK_SUCCESS;
17213	}
17214	(allocator->GetVulkanFunctions().vkDestroyBuffer)(allocator->m_hDevice, pBuffer, allocator->GetAllocationCallbacks());
17215	}
17216	return res;
17217	}
17218
17219	VMA_CALL_PRE void VMA_CALL_POST vmaDestroyBuffer(
17220	VmaAllocator allocator,
17221	VkBuffer buffer,
17222	VmaAllocation allocation)
17223	{
17224	VMA_ASSERT(allocator);
17225
17226	if(buffer == VK_NULL_HANDLE && allocation == VK_NULL_HANDLE)
17227	{
17228	return;
17229	}
17230
17231	VMA_DEBUG_LOG("vmaDestroyBuffer");
17232
17233	VMA_DEBUG_GLOBAL_MUTEX_LOCK
17234
17235	if(buffer != VK_NULL_HANDLE)
17236	{
17237	(*allocator->GetVulkanFunctions().vkDestroyBuffer)(allocator->m_hDevice, buffer, allocator->GetAllocationCallbacks());
17238	}
17239
17240	if(allocation != VK_NULL_HANDLE)
17241	{
17242	allocator->FreeMemory(
17243	`1`, // allocationCount
17244	&allocation);
17245	}
17246	}
17247
17248	VMA_CALL_PRE VkResult VMA_CALL_POST vmaCreateImage(
17249	VmaAllocator allocator,
17250	const VkImageCreateInfo* pImageCreateInfo,
17251	const VmaAllocationCreateInfo* pAllocationCreateInfo,
17252	VkImage* pImage,
17253	VmaAllocation* pAllocation,
17254	VmaAllocationInfo* pAllocationInfo)
17255	{
17256	VMA_ASSERT(allocator && pImageCreateInfo && pAllocationCreateInfo && pImage && pAllocation);
17257
17258	if(pImageCreateInfo->extent.width == `0` \|\|
17259	pImageCreateInfo->extent.height == `0` \|\|
17260	pImageCreateInfo->extent.depth == `0` \|\|
17261	pImageCreateInfo->mipLevels == `0` \|\|
17262	pImageCreateInfo->arrayLayers == `0`)
17263	{
17264	return VK_ERROR_INITIALIZATION_FAILED;
17265	}
17266
17267	VMA_DEBUG_LOG("vmaCreateImage");
17268
17269	VMA_DEBUG_GLOBAL_MUTEX_LOCK
17270
17271	*pImage = VK_NULL_HANDLE;
17272	*pAllocation = VK_NULL_HANDLE;
17273
17274	// 1. Create VkImage.
17275	VkResult res = (*allocator->GetVulkanFunctions().vkCreateImage)(
17276	allocator->m_hDevice,
17277	pImageCreateInfo,
17278	allocator->GetAllocationCallbacks(),
17279	pImage);
17280	if(res >= `0`)
17281	{
17282	VmaSuballocationType suballocType = pImageCreateInfo->tiling == VK_IMAGE_TILING_OPTIMAL ?
17283	VMA_SUBALLOCATION_TYPE_IMAGE_OPTIMAL :
17284	VMA_SUBALLOCATION_TYPE_IMAGE_LINEAR;
17285
17286	// 2. Allocate memory using allocator.
17287	VkMemoryRequirements vkMemReq = {};
17288	bool requiresDedicatedAllocation = false;
17289	bool prefersDedicatedAllocation = false;
17290	allocator->GetImageMemoryRequirements(*pImage, vkMemReq,
17291	requiresDedicatedAllocation, prefersDedicatedAllocation);
17292
17293	res = allocator->AllocateMemory(
17294	vkMemReq,
17295	requiresDedicatedAllocation,
17296	prefersDedicatedAllocation,
17297	VK_NULL_HANDLE, // dedicatedBuffer
17298	pImage, // dedicatedImage*
17299	pImageCreateInfo->usage, // dedicatedBufferImageUsage
17300	*pAllocationCreateInfo,
17301	suballocType,
17302	`1`, // allocationCount
17303	pAllocation);
17304
17305	if(res >= `0`)
17306	{
17307	// 3. Bind image with memory.
17308	if((pAllocationCreateInfo->flags & VMA_ALLOCATION_CREATE_DONT_BIND_BIT) == `0`)
17309	{
17310	res = allocator->BindImageMemory(pAllocation, `0`, pImage, VMA_NULL);
17311	}
17312	if(res >= `0`)
17313	{
17314	// All steps succeeded.
17315	#if VMA_STATS_STRING_ENABLED
17316	(*pAllocation)->InitBufferImageUsage(pImageCreateInfo->usage);
17317	#endif
17318	if(pAllocationInfo != VMA_NULL)
17319	{
17320	allocator->GetAllocationInfo(*pAllocation, pAllocationInfo);
17321	}
17322
17323	return VK_SUCCESS;
17324	}
17325	allocator->FreeMemory(
17326	`1`, // allocationCount
17327	pAllocation);
17328	*pAllocation = VK_NULL_HANDLE;
17329	(allocator->GetVulkanFunctions().vkDestroyImage)(allocator->m_hDevice, pImage, allocator->GetAllocationCallbacks());
17330	*pImage = VK_NULL_HANDLE;
17331	return res;
17332	}
17333	(allocator->GetVulkanFunctions().vkDestroyImage)(allocator->m_hDevice, pImage, allocator->GetAllocationCallbacks());
17334	*pImage = VK_NULL_HANDLE;
17335	return res;
17336	}
17337	return res;
17338	}
17339
17340	VMA_CALL_PRE VkResult VMA_CALL_POST vmaCreateAliasingImage(
17341	VmaAllocator VMA_NOT_NULL allocator,
17342	VmaAllocation VMA_NOT_NULL allocation,
17343	const VkImageCreateInfo* VMA_NOT_NULL pImageCreateInfo,
17344	VkImage VMA_NULLABLE_NON_DISPATCHABLE* VMA_NOT_NULL pImage)
17345	{
17346	VMA_ASSERT(allocator && pImageCreateInfo && pImage && allocation);
17347
17348	*pImage = VK_NULL_HANDLE;
17349
17350	VMA_DEBUG_LOG("vmaCreateImage");
17351
17352	if (pImageCreateInfo->extent.width == `0` \|\|
17353	pImageCreateInfo->extent.height == `0` \|\|
17354	pImageCreateInfo->extent.depth == `0` \|\|
17355	pImageCreateInfo->mipLevels == `0` \|\|
17356	pImageCreateInfo->arrayLayers == `0`)
17357	{
17358	return VK_ERROR_INITIALIZATION_FAILED;
17359	}
17360
17361	VMA_DEBUG_GLOBAL_MUTEX_LOCK
17362
17363	// 1. Create VkImage.
17364	VkResult res = (*allocator->GetVulkanFunctions().vkCreateImage)(
17365	allocator->m_hDevice,
17366	pImageCreateInfo,
17367	allocator->GetAllocationCallbacks(),
17368	pImage);
17369	if (res >= `0`)
17370	{
17371	// 2. Bind image with memory.
17372	res = allocator->BindImageMemory(allocation, `0`, *pImage, VMA_NULL);
17373	if (res >= `0`)
17374	{
17375	return VK_SUCCESS;
17376	}
17377	(allocator->GetVulkanFunctions().vkDestroyImage)(allocator->m_hDevice, pImage, allocator->GetAllocationCallbacks());
17378	}
17379	return res;
17380	}
17381
17382	VMA_CALL_PRE void VMA_CALL_POST vmaDestroyImage(
17383	VmaAllocator VMA_NOT_NULL allocator,
17384	VkImage VMA_NULLABLE_NON_DISPATCHABLE image,
17385	VmaAllocation VMA_NULLABLE allocation)
17386	{
17387	VMA_ASSERT(allocator);
17388
17389	if(image == VK_NULL_HANDLE && allocation == VK_NULL_HANDLE)
17390	{
17391	return;
17392	}
17393
17394	VMA_DEBUG_LOG("vmaDestroyImage");
17395
17396	VMA_DEBUG_GLOBAL_MUTEX_LOCK
17397
17398	if(image != VK_NULL_HANDLE)
17399	{
17400	(*allocator->GetVulkanFunctions().vkDestroyImage)(allocator->m_hDevice, image, allocator->GetAllocationCallbacks());
17401	}
17402	if(allocation != VK_NULL_HANDLE)
17403	{
17404	allocator->FreeMemory(
17405	`1`, // allocationCount
17406	&allocation);
17407	}
17408	}
17409
17410	VMA_CALL_PRE VkResult VMA_CALL_POST vmaCreateVirtualBlock(
17411	const VmaVirtualBlockCreateInfo* VMA_NOT_NULL pCreateInfo,
17412	VmaVirtualBlock VMA_NULLABLE * VMA_NOT_NULL pVirtualBlock)
17413	{
17414	VMA_ASSERT(pCreateInfo && pVirtualBlock);
17415	VMA_ASSERT(pCreateInfo->size > `0`);
17416	VMA_DEBUG_LOG("vmaCreateVirtualBlock");
17417	VMA_DEBUG_GLOBAL_MUTEX_LOCK;
17418	pVirtualBlock = vma_new(pCreateInfo->pAllocationCallbacks, VmaVirtualBlock_T)(pCreateInfo);
17419	VkResult res = (*pVirtualBlock)->Init();
17420	if(res < `0`)
17421	{
17422	vma_delete(pCreateInfo->pAllocationCallbacks, *pVirtualBlock);
17423	*pVirtualBlock = VK_NULL_HANDLE;
17424	}
17425	return res;
17426	}
17427
17428	VMA_CALL_PRE void VMA_CALL_POST vmaDestroyVirtualBlock(VmaVirtualBlock VMA_NULLABLE virtualBlock)
17429	{
17430	if(virtualBlock != VK_NULL_HANDLE)
17431	{
17432	VMA_DEBUG_LOG("vmaDestroyVirtualBlock");
17433	VMA_DEBUG_GLOBAL_MUTEX_LOCK;
17434	VkAllocationCallbacks allocationCallbacks = virtualBlock->m_AllocationCallbacks; // Have to copy the callbacks when destroying.
17435	vma_delete(&allocationCallbacks, virtualBlock);
17436	}
17437	}
17438
17439	VMA_CALL_PRE VkBool32 VMA_CALL_POST vmaIsVirtualBlockEmpty(VmaVirtualBlock VMA_NOT_NULL virtualBlock)
17440	{
17441	VMA_ASSERT(virtualBlock != VK_NULL_HANDLE);
17442	VMA_DEBUG_LOG("vmaIsVirtualBlockEmpty");
17443	VMA_DEBUG_GLOBAL_MUTEX_LOCK;
17444	return virtualBlock->IsEmpty() ? VK_TRUE : VK_FALSE;
17445	}
17446
17447	VMA_CALL_PRE void VMA_CALL_POST vmaGetVirtualAllocationInfo(VmaVirtualBlock VMA_NOT_NULL virtualBlock,
17448	VmaVirtualAllocation VMA_NOT_NULL_NON_DISPATCHABLE allocation, VmaVirtualAllocationInfo* VMA_NOT_NULL pVirtualAllocInfo)
17449	{
17450	VMA_ASSERT(virtualBlock != VK_NULL_HANDLE && pVirtualAllocInfo != VMA_NULL);
17451	VMA_DEBUG_LOG("vmaGetVirtualAllocationInfo");
17452	VMA_DEBUG_GLOBAL_MUTEX_LOCK;
17453	virtualBlock->GetAllocationInfo(allocation, *pVirtualAllocInfo);
17454	}
17455
17456	VMA_CALL_PRE VkResult VMA_CALL_POST vmaVirtualAllocate(VmaVirtualBlock VMA_NOT_NULL virtualBlock,
17457	const VmaVirtualAllocationCreateInfo* VMA_NOT_NULL pCreateInfo, VmaVirtualAllocation VMA_NULLABLE_NON_DISPATCHABLE* VMA_NOT_NULL pAllocation,
17458	VkDeviceSize* VMA_NULLABLE pOffset)
17459	{
17460	VMA_ASSERT(virtualBlock != VK_NULL_HANDLE && pCreateInfo != VMA_NULL && pAllocation != VMA_NULL);
17461	VMA_DEBUG_LOG("vmaVirtualAllocate");
17462	VMA_DEBUG_GLOBAL_MUTEX_LOCK;
17463	return virtualBlock->Allocate(pCreateInfo, pAllocation, pOffset);
17464	}
17465
17466	VMA_CALL_PRE void VMA_CALL_POST vmaVirtualFree(VmaVirtualBlock VMA_NOT_NULL virtualBlock, VmaVirtualAllocation VMA_NULLABLE_NON_DISPATCHABLE allocation)
17467	{
17468	if(allocation != VK_NULL_HANDLE)
17469	{
17470	VMA_ASSERT(virtualBlock != VK_NULL_HANDLE);
17471	VMA_DEBUG_LOG("vmaVirtualFree");
17472	VMA_DEBUG_GLOBAL_MUTEX_LOCK;
17473	virtualBlock->Free(allocation);
17474	}
17475	}
17476
17477	VMA_CALL_PRE void VMA_CALL_POST vmaClearVirtualBlock(VmaVirtualBlock VMA_NOT_NULL virtualBlock)
17478	{
17479	VMA_ASSERT(virtualBlock != VK_NULL_HANDLE);
17480	VMA_DEBUG_LOG("vmaClearVirtualBlock");
17481	VMA_DEBUG_GLOBAL_MUTEX_LOCK;
17482	virtualBlock->Clear();
17483	}
17484
17485	VMA_CALL_PRE void VMA_CALL_POST vmaSetVirtualAllocationUserData(VmaVirtualBlock VMA_NOT_NULL virtualBlock,
17486	VmaVirtualAllocation VMA_NOT_NULL_NON_DISPATCHABLE allocation, void* VMA_NULLABLE pUserData)
17487	{
17488	VMA_ASSERT(virtualBlock != VK_NULL_HANDLE);
17489	VMA_DEBUG_LOG("vmaSetVirtualAllocationUserData");
17490	VMA_DEBUG_GLOBAL_MUTEX_LOCK;
17491	virtualBlock->SetAllocationUserData(allocation, pUserData);
17492	}
17493
17494	VMA_CALL_PRE void VMA_CALL_POST vmaGetVirtualBlockStatistics(VmaVirtualBlock VMA_NOT_NULL virtualBlock,
17495	VmaStatistics* VMA_NOT_NULL pStats)
17496	{
17497	VMA_ASSERT(virtualBlock != VK_NULL_HANDLE && pStats != VMA_NULL);
17498	VMA_DEBUG_LOG("vmaGetVirtualBlockStatistics");
17499	VMA_DEBUG_GLOBAL_MUTEX_LOCK;
17500	virtualBlock->GetStatistics(*pStats);
17501	}
17502
17503	VMA_CALL_PRE void VMA_CALL_POST vmaCalculateVirtualBlockStatistics(VmaVirtualBlock VMA_NOT_NULL virtualBlock,
17504	VmaDetailedStatistics* VMA_NOT_NULL pStats)
17505	{
17506	VMA_ASSERT(virtualBlock != VK_NULL_HANDLE && pStats != VMA_NULL);
17507	VMA_DEBUG_LOG("vmaCalculateVirtualBlockStatistics");
17508	VMA_DEBUG_GLOBAL_MUTEX_LOCK;
17509	virtualBlock->CalculateDetailedStatistics(*pStats);
17510	}
17511
17512	#if VMA_STATS_STRING_ENABLED
17513
17514	VMA_CALL_PRE void VMA_CALL_POST vmaBuildVirtualBlockStatsString(VmaVirtualBlock VMA_NOT_NULL virtualBlock,
17515	char* VMA_NULLABLE * VMA_NOT_NULL ppStatsString, VkBool32 detailedMap)
17516	{
17517	VMA_ASSERT(virtualBlock != VK_NULL_HANDLE && ppStatsString != VMA_NULL);
17518	VMA_DEBUG_GLOBAL_MUTEX_LOCK;
17519	const VkAllocationCallbacks* allocationCallbacks = virtualBlock->GetAllocationCallbacks();
17520	VmaStringBuilder sb(allocationCallbacks);
17521	virtualBlock->BuildStatsString(detailedMap != VK_FALSE, sb);
17522	*ppStatsString = VmaCreateStringCopy(allocationCallbacks, sb.GetData(), sb.GetLength());
17523	}
17524
17525	VMA_CALL_PRE void VMA_CALL_POST vmaFreeVirtualBlockStatsString(VmaVirtualBlock VMA_NOT_NULL virtualBlock,
17526	char* VMA_NULLABLE pStatsString)
17527	{
17528	if(pStatsString != VMA_NULL)
17529	{
17530	VMA_ASSERT(virtualBlock != VK_NULL_HANDLE);
17531	VMA_DEBUG_GLOBAL_MUTEX_LOCK;
17532	VmaFreeString(virtualBlock->GetAllocationCallbacks(), pStatsString);
17533	}
17534	}
17535	#endif // VMA_STATS_STRING_ENABLED
17536	#endif // _VMA_PUBLIC_INTERFACE
17537	#endif // VMA_IMPLEMENTATION
17538
17539	/**
17540	\page quick_start Quick start
17541
17542	\section quick_start_project_setup Project setup
17543
17544	Vulkan Memory Allocator comes in form of a "stb-style" single header file.
17545	You don't need to build it as a separate library project.
17546	You can add this file directly to your project and submit it to code repository next to your other source files.
17547
17548	"Single header" doesn't mean that everything is contained in C/C++ declarations,
17549	like it tends to be in case of inline functions or C++ templates.
17550	It means that implementation is bundled with interface in a single file and needs to be extracted using preprocessor macro.
17551	If you don't do it properly, you will get linker errors.
17552
17553	To do it properly:
17554
17555	-# Include "vk_mem_alloc.h" file in each CPP file where you want to use the library.
17556	This includes declarations of all members of the library.
17557	-# In exactly one CPP file define following macro before this include.
17558	It enables also internal definitions.
17559
17560	\code
17561	#define VMA_IMPLEMENTATION
17562	#include "vk_mem_alloc.h"
17563	\endcode
17564
17565	It may be a good idea to create dedicated CPP file just for this purpose.
17566
17567	This library includes header `<vulkan/vulkan.h>`, which in turn
17568	includes `<windows.h>` on Windows. If you need some specific macros defined
17569	before including these headers (like `WIN32_LEAN_AND_MEAN` or
17570	`WINVER` for Windows, `VK_USE_PLATFORM_WIN32_KHR` for Vulkan), you must define
17571	them before every `#include` of this library.
17572
17573	This library is written in C++, but has C-compatible interface.
17574	Thus you can include and use vk_mem_alloc.h in C or C++ code, but full
17575	implementation with `VMA_IMPLEMENTATION` macro must be compiled as C++, NOT as C.
17576	Some features of C++14 used. STL containers, RTTI, or C++ exceptions are not used.
17577
17578
17579	\section quick_start_initialization Initialization
17580
17581	At program startup:
17582
17583	-# Initialize Vulkan to have `VkPhysicalDevice`, `VkDevice` and `VkInstance` object.
17584	-# Fill VmaAllocatorCreateInfo structure and create #VmaAllocator object by
17585	calling vmaCreateAllocator().
17586
17587	Only members `physicalDevice`, `device`, `instance` are required.
17588	However, you should inform the library which Vulkan version do you use by setting
17589	VmaAllocatorCreateInfo::vulkanApiVersion and which extensions did you enable
17590	by setting VmaAllocatorCreateInfo::flags (like #VMA_ALLOCATOR_CREATE_BUFFER_DEVICE_ADDRESS_BIT for VK_KHR_buffer_device_address).
17591	Otherwise, VMA would use only features of Vulkan 1.0 core with no extensions.
17592
17593	You may need to configure importing Vulkan functions. There are 3 ways to do this:
17594
17595	-# If you link with Vulkan static library* (e.g. "vulkan-1.lib" on Windows):*
17596	- You don't need to do anything.
17597	- VMA will use these, as macro `VMA_STATIC_VULKAN_FUNCTIONS` is defined to 1 by default.
17598	-# If you want VMA to fetch pointers to Vulkan functions dynamically* using `vkGetInstanceProcAddr`,*
17599	`vkGetDeviceProcAddr` (this is the option presented in the example below):
17600	- Define `VMA_STATIC_VULKAN_FUNCTIONS` to 0, `VMA_DYNAMIC_VULKAN_FUNCTIONS` to 1.
17601	- Provide pointers to these two functions via VmaVulkanFunctions::vkGetInstanceProcAddr,
17602	VmaVulkanFunctions::vkGetDeviceProcAddr.
17603	- The library will fetch pointers to all other functions it needs internally.
17604	-# If you fetch pointers to all Vulkan functions in a custom way, e.g. using some loader like
17605	[Volk](https://github.com/zeux/volk):
17606	- Define `VMA_STATIC_VULKAN_FUNCTIONS` and `VMA_DYNAMIC_VULKAN_FUNCTIONS` to 0.
17607	- Pass these pointers via structure #VmaVulkanFunctions.
17608
17609	\code
17610	VmaVulkanFunctions vulkanFunctions = {};
17611	vulkanFunctions.vkGetInstanceProcAddr = &vkGetInstanceProcAddr;
17612	vulkanFunctions.vkGetDeviceProcAddr = &vkGetDeviceProcAddr;
17613
17614	VmaAllocatorCreateInfo allocatorCreateInfo = {};
17615	allocatorCreateInfo.vulkanApiVersion = VK_API_VERSION_1_2;
17616	allocatorCreateInfo.physicalDevice = physicalDevice;
17617	allocatorCreateInfo.device = device;
17618	allocatorCreateInfo.instance = instance;
17619	allocatorCreateInfo.pVulkanFunctions = &vulkanFunctions;
17620
17621	VmaAllocator allocator;
17622	vmaCreateAllocator(&allocatorCreateInfo, &allocator);
17623	\endcode
17624
17625
17626	\section quick_start_resource_allocation Resource allocation
17627
17628	When you want to create a buffer or image:
17629
17630	-# Fill `VkBufferCreateInfo` / `VkImageCreateInfo` structure.
17631	-# Fill VmaAllocationCreateInfo structure.
17632	-# Call vmaCreateBuffer() / vmaCreateImage() to get `VkBuffer`/`VkImage` with memory
17633	already allocated and bound to it, plus #VmaAllocation objects that represents its underlying memory.
17634
17635	\code
17636	VkBufferCreateInfo bufferInfo = { VK_STRUCTURE_TYPE_BUFFER_CREATE_INFO };
17637	bufferInfo.size = 65536;
17638	bufferInfo.usage = VK_BUFFER_USAGE_VERTEX_BUFFER_BIT \| VK_BUFFER_USAGE_TRANSFER_DST_BIT;
17639
17640	VmaAllocationCreateInfo allocInfo = {};
17641	allocInfo.usage = VMA_MEMORY_USAGE_AUTO;
17642
17643	VkBuffer buffer;
17644	VmaAllocation allocation;
17645	vmaCreateBuffer(allocator, &bufferInfo, &allocInfo, &buffer, &allocation, nullptr);
17646	\endcode
17647
17648	Don't forget to destroy your objects when no longer needed:
17649
17650	\code
17651	vmaDestroyBuffer(allocator, buffer, allocation);
17652	vmaDestroyAllocator(allocator);
17653	\endcode
17654
17655
17656	\page choosing_memory_type Choosing memory type
17657
17658	Physical devices in Vulkan support various combinations of memory heaps and
17659	types. Help with choosing correct and optimal memory type for your specific
17660	resource is one of the key features of this library. You can use it by filling
17661	appropriate members of VmaAllocationCreateInfo structure, as described below.
17662	You can also combine multiple methods.
17663
17664	-# If you just want to find memory type index that meets your requirements, you
17665	can use function: vmaFindMemoryTypeIndexForBufferInfo(),
17666	vmaFindMemoryTypeIndexForImageInfo(), vmaFindMemoryTypeIndex().
17667	-# If you want to allocate a region of device memory without association with any
17668	specific image or buffer, you can use function vmaAllocateMemory(). Usage of
17669	this function is not recommended and usually not needed.
17670	vmaAllocateMemoryPages() function is also provided for creating multiple allocations at once,
17671	which may be useful for sparse binding.
17672	-# If you already have a buffer or an image created, you want to allocate memory
17673	for it and then you will bind it yourself, you can use function
17674	vmaAllocateMemoryForBuffer(), vmaAllocateMemoryForImage().
17675	For binding you should use functions: vmaBindBufferMemory(), vmaBindImageMemory()
17676	or their extended versions: vmaBindBufferMemory2(), vmaBindImageMemory2().
17677	-# This is the easiest and recommended way to use this library:
17678	If you want to create a buffer or an image, allocate memory for it and bind
17679	them together, all in one call, you can use function vmaCreateBuffer(),
17680	vmaCreateImage().
17681
17682	When using 3. or 4., the library internally queries Vulkan for memory types
17683	supported for that buffer or image (function `vkGetBufferMemoryRequirements()`)
17684	and uses only one of these types.
17685
17686	If no memory type can be found that meets all the requirements, these functions
17687	return `VK_ERROR_FEATURE_NOT_PRESENT`.
17688
17689	You can leave VmaAllocationCreateInfo structure completely filled with zeros.
17690	It means no requirements are specified for memory type.
17691	It is valid, although not very useful.
17692
17693	\section choosing_memory_type_usage Usage
17694
17695	The easiest way to specify memory requirements is to fill member
17696	VmaAllocationCreateInfo::usage using one of the values of enum #VmaMemoryUsage.
17697	It defines high level, common usage types.
17698	Since version 3 of the library, it is recommended to use #VMA_MEMORY_USAGE_AUTO to let it select best memory type for your resource automatically.
17699
17700	For example, if you want to create a uniform buffer that will be filled using
17701	transfer only once or infrequently and then used for rendering every frame as a uniform buffer, you can
17702	do it using following code. The buffer will most likely end up in a memory type with
17703	`VK_MEMORY_PROPERTY_DEVICE_LOCAL_BIT` to be fast to access by the GPU device.
17704
17705	\code
17706	VkBufferCreateInfo bufferInfo = { VK_STRUCTURE_TYPE_BUFFER_CREATE_INFO };
17707	bufferInfo.size = 65536;
17708	bufferInfo.usage = VK_BUFFER_USAGE_UNIFORM_BUFFER_BIT \| VK_BUFFER_USAGE_TRANSFER_DST_BIT;
17709
17710	VmaAllocationCreateInfo allocInfo = {};
17711	allocInfo.usage = VMA_MEMORY_USAGE_AUTO;
17712
17713	VkBuffer buffer;
17714	VmaAllocation allocation;
17715	vmaCreateBuffer(allocator, &bufferInfo, &allocInfo, &buffer, &allocation, nullptr);
17716	\endcode
17717
17718	If you have a preference for putting the resource in GPU (device) memory or CPU (host) memory
17719	on systems with discrete graphics card that have the memories separate, you can use
17720	#VMA_MEMORY_USAGE_AUTO_PREFER_DEVICE or #VMA_MEMORY_USAGE_AUTO_PREFER_HOST.
17721
17722	When using `VMA_MEMORY_USAGE_AUTO` while you want to map the allocated memory,*
17723	you also need to specify one of the host access flags:
17724	#VMA_ALLOCATION_CREATE_HOST_ACCESS_SEQUENTIAL_WRITE_BIT or #VMA_ALLOCATION_CREATE_HOST_ACCESS_RANDOM_BIT.
17725	This will help the library decide about preferred memory type to ensure it has `VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT`
17726	so you can map it.
17727
17728	For example, a staging buffer that will be filled via mapped pointer and then
17729	used as a source of transfer to the buffer decribed previously can be created like this.
17730	It will likely and up in a memory type that is `HOST_VISIBLE` and `HOST_COHERENT`
17731	but not `HOST_CACHED` (meaning uncached, write-combined) and not `DEVICE_LOCAL` (meaning system RAM).
17732
17733	\code
17734	VkBufferCreateInfo stagingBufferInfo = { VK_STRUCTURE_TYPE_BUFFER_CREATE_INFO };
17735	stagingBufferInfo.size = 65536;
17736	stagingBufferInfo.usage = VK_BUFFER_USAGE_TRANSFER_SRC_BIT;
17737
17738	VmaAllocationCreateInfo stagingAllocInfo = {};
17739	stagingAllocInfo.usage = VMA_MEMORY_USAGE_AUTO;
17740	stagingAllocInfo.flags = VMA_ALLOCATION_CREATE_HOST_ACCESS_SEQUENTIAL_WRITE_BIT;
17741
17742	VkBuffer stagingBuffer;
17743	VmaAllocation stagingAllocation;
17744	vmaCreateBuffer(allocator, &stagingBufferInfo, &stagingAllocInfo, &stagingBuffer, &stagingAllocation, nullptr);
17745	\endcode
17746
17747	For more examples of creating different kinds of resources, see chapter \ref usage_patterns.
17748
17749	Usage values `VMA_MEMORY_USAGE_AUTO` are legal to use only when the library knows*
17750	about the resource being created by having `VkBufferCreateInfo` / `VkImageCreateInfo` passed,
17751	so they work with functions like: vmaCreateBuffer(), vmaCreateImage(), vmaFindMemoryTypeIndexForBufferInfo() etc.
17752	If you allocate raw memory using function vmaAllocateMemory(), you have to use other means of selecting
17753	memory type, as decribed below.
17754
17755	\note
17756	Old usage values (`VMA_MEMORY_USAGE_GPU_ONLY`, `VMA_MEMORY_USAGE_CPU_ONLY`,
17757	`VMA_MEMORY_USAGE_CPU_TO_GPU`, `VMA_MEMORY_USAGE_GPU_TO_CPU`, `VMA_MEMORY_USAGE_CPU_COPY`)
17758	are still available and work same way as in previous versions of the library
17759	for backward compatibility, but they are not recommended.
17760
17761	\section choosing_memory_type_required_preferred_flags Required and preferred flags
17762
17763	You can specify more detailed requirements by filling members
17764	VmaAllocationCreateInfo::requiredFlags and VmaAllocationCreateInfo::preferredFlags
17765	with a combination of bits from enum `VkMemoryPropertyFlags`. For example,
17766	if you want to create a buffer that will be persistently mapped on host (so it
17767	must be `HOST_VISIBLE`) and preferably will also be `HOST_COHERENT` and `HOST_CACHED`,
17768	use following code:
17769
17770	\code
17771	VmaAllocationCreateInfo allocInfo = {};
17772	allocInfo.requiredFlags = VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT;
17773	allocInfo.preferredFlags = VK_MEMORY_PROPERTY_HOST_COHERENT_BIT \| VK_MEMORY_PROPERTY_HOST_CACHED_BIT;
17774	allocInfo.flags = VMA_ALLOCATION_CREATE_HOST_ACCESS_RANDOM_BIT \| VMA_ALLOCATION_CREATE_MAPPED_BIT;
17775
17776	VkBuffer buffer;
17777	VmaAllocation allocation;
17778	vmaCreateBuffer(allocator, &bufferInfo, &allocInfo, &buffer, &allocation, nullptr);
17779	\endcode
17780
17781	A memory type is chosen that has all the required flags and as many preferred
17782	flags set as possible.
17783
17784	Value passed in VmaAllocationCreateInfo::usage is internally converted to a set of required and preferred flags,
17785	plus some extra "magic" (heuristics).
17786
17787	\section choosing_memory_type_explicit_memory_types Explicit memory types
17788
17789	If you inspected memory types available on the physical device and you have
17790	a preference for memory types that you want to use, you can fill member
17791	VmaAllocationCreateInfo::memoryTypeBits. It is a bit mask, where each bit set
17792	means that a memory type with that index is allowed to be used for the
17793	allocation. Special value 0, just like `UINT32_MAX`, means there are no
17794	restrictions to memory type index.
17795
17796	Please note that this member is NOT just a memory type index.
17797	Still you can use it to choose just one, specific memory type.
17798	For example, if you already determined that your buffer should be created in
17799	memory type 2, use following code:
17800
17801	\code
17802	uint32_t memoryTypeIndex = 2;
17803
17804	VmaAllocationCreateInfo allocInfo = {};
17805	allocInfo.memoryTypeBits = 1u << memoryTypeIndex;
17806
17807	VkBuffer buffer;
17808	VmaAllocation allocation;
17809	vmaCreateBuffer(allocator, &bufferInfo, &allocInfo, &buffer, &allocation, nullptr);
17810	\endcode
17811
17812
17813	\section choosing_memory_type_custom_memory_pools Custom memory pools
17814
17815	If you allocate from custom memory pool, all the ways of specifying memory
17816	requirements described above are not applicable and the aforementioned members
17817	of VmaAllocationCreateInfo structure are ignored. Memory type is selected
17818	explicitly when creating the pool and then used to make all the allocations from
17819	that pool. For further details, see \ref custom_memory_pools.
17820
17821	\section choosing_memory_type_dedicated_allocations Dedicated allocations
17822
17823	Memory for allocations is reserved out of larger block of `VkDeviceMemory`
17824	allocated from Vulkan internally. That is the main feature of this whole library.
17825	You can still request a separate memory block to be created for an allocation,
17826	just like you would do in a trivial solution without using any allocator.
17827	In that case, a buffer or image is always bound to that memory at offset 0.
17828	This is called a "dedicated allocation".
17829	You can explicitly request it by using flag #VMA_ALLOCATION_CREATE_DEDICATED_MEMORY_BIT.
17830	The library can also internally decide to use dedicated allocation in some cases, e.g.:
17831
17832	- When the size of the allocation is large.
17833	- When [VK_KHR_dedicated_allocation](@ref vk_khr_dedicated_allocation) extension is enabled
17834	and it reports that dedicated allocation is required or recommended for the resource.
17835	- When allocation of next big memory block fails due to not enough device memory,
17836	but allocation with the exact requested size succeeds.
17837
17838
17839	\page memory_mapping Memory mapping
17840
17841	To "map memory" in Vulkan means to obtain a CPU pointer to `VkDeviceMemory`,
17842	to be able to read from it or write to it in CPU code.
17843	Mapping is possible only of memory allocated from a memory type that has
17844	`VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT` flag.
17845	Functions `vkMapMemory()`, `vkUnmapMemory()` are designed for this purpose.
17846	You can use them directly with memory allocated by this library,
17847	but it is not recommended because of following issue:
17848	Mapping the same `VkDeviceMemory` block multiple times is illegal - only one mapping at a time is allowed.
17849	This includes mapping disjoint regions. Mapping is not reference-counted internally by Vulkan.
17850	Because of this, Vulkan Memory Allocator provides following facilities:
17851
17852	\note If you want to be able to map an allocation, you need to specify one of the flags
17853	#VMA_ALLOCATION_CREATE_HOST_ACCESS_SEQUENTIAL_WRITE_BIT or #VMA_ALLOCATION_CREATE_HOST_ACCESS_RANDOM_BIT
17854	in VmaAllocationCreateInfo::flags. These flags are required for an allocation to be mappable
17855	when using #VMA_MEMORY_USAGE_AUTO or other `VMA_MEMORY_USAGE_AUTO` enum values.*
17856	For other usage values they are ignored and every such allocation made in `HOST_VISIBLE` memory type is mappable,
17857	but they can still be used for consistency.
17858
17859	\section memory_mapping_mapping_functions Mapping functions
17860
17861	The library provides following functions for mapping of a specific #VmaAllocation: vmaMapMemory(), vmaUnmapMemory().
17862	They are safer and more convenient to use than standard Vulkan functions.
17863	You can map an allocation multiple times simultaneously - mapping is reference-counted internally.
17864	You can also map different allocations simultaneously regardless of whether they use the same `VkDeviceMemory` block.
17865	The way it is implemented is that the library always maps entire memory block, not just region of the allocation.
17866	For further details, see description of vmaMapMemory() function.
17867	Example:
17868
17869	\code
17870	// Having these objects initialized:
17871	struct ConstantBuffer
17872	{
17873	...
17874	};
17875	ConstantBuffer constantBufferData = ...
17876
17877	VmaAllocator allocator = ...
17878	VkBuffer constantBuffer = ...
17879	VmaAllocation constantBufferAllocation = ...
17880
17881	// You can map and fill your buffer using following code:
17882
17883	void mappedData;*
17884	vmaMapMemory(allocator, constantBufferAllocation, &mappedData);
17885	memcpy(mappedData, &constantBufferData, sizeof(constantBufferData));
17886	vmaUnmapMemory(allocator, constantBufferAllocation);
17887	\endcode
17888
17889	When mapping, you may see a warning from Vulkan validation layer similar to this one:
17890
17891	<i>Mapping an image with layout VK_IMAGE_LAYOUT_DEPTH_STENCIL_ATTACHMENT_OPTIMAL can result in undefined behavior if this memory is used by the device. Only GENERAL or PREINITIALIZED should be used.</i>
17892
17893	It happens because the library maps entire `VkDeviceMemory` block, where different
17894	types of images and buffers may end up together, especially on GPUs with unified memory like Intel.
17895	You can safely ignore it if you are sure you access only memory of the intended
17896	object that you wanted to map.
17897
17898
17899	\section memory_mapping_persistently_mapped_memory Persistently mapped memory
17900
17901	Kepping your memory persistently mapped is generally OK in Vulkan.
17902	You don't need to unmap it before using its data on the GPU.
17903	The library provides a special feature designed for that:
17904	Allocations made with #VMA_ALLOCATION_CREATE_MAPPED_BIT flag set in
17905	VmaAllocationCreateInfo::flags stay mapped all the time,
17906	so you can just access CPU pointer to it any time
17907	without a need to call any "map" or "unmap" function.
17908	Example:
17909
17910	\code
17911	VkBufferCreateInfo bufCreateInfo = { VK_STRUCTURE_TYPE_BUFFER_CREATE_INFO };
17912	bufCreateInfo.size = sizeof(ConstantBuffer);
17913	bufCreateInfo.usage = VK_BUFFER_USAGE_TRANSFER_SRC_BIT;
17914
17915	VmaAllocationCreateInfo allocCreateInfo = {};
17916	allocCreateInfo.usage = VMA_MEMORY_USAGE_AUTO;
17917	allocCreateInfo.flags = VMA_ALLOCATION_CREATE_HOST_ACCESS_SEQUENTIAL_WRITE_BIT \|
17918	VMA_ALLOCATION_CREATE_MAPPED_BIT;
17919
17920	VkBuffer buf;
17921	VmaAllocation alloc;
17922	VmaAllocationInfo allocInfo;
17923	vmaCreateBuffer(allocator, &bufCreateInfo, &allocCreateInfo, &buf, &alloc, &allocInfo);
17924
17925	// Buffer is already mapped. You can access its memory.
17926	memcpy(allocInfo.pMappedData, &constantBufferData, sizeof(constantBufferData));
17927	\endcode
17928
17929	\note #VMA_ALLOCATION_CREATE_MAPPED_BIT by itself doesn't guarantee that the allocation will end up
17930	in a mappable memory type.
17931	For this, you need to also specify #VMA_ALLOCATION_CREATE_HOST_ACCESS_SEQUENTIAL_WRITE_BIT or
17932	#VMA_ALLOCATION_CREATE_HOST_ACCESS_RANDOM_BIT.
17933	#VMA_ALLOCATION_CREATE_MAPPED_BIT only guarantees that if the memory is `HOST_VISIBLE`, the allocation will be mapped on creation.
17934	For an example of how to make use of this fact, see section \ref usage_patterns_advanced_data_uploading.
17935
17936	\section memory_mapping_cache_control Cache flush and invalidate
17937
17938	Memory in Vulkan doesn't need to be unmapped before using it on GPU,
17939	but unless a memory types has `VK_MEMORY_PROPERTY_HOST_COHERENT_BIT` flag set,
17940	you need to manually invalidate* cache before reading of mapped pointer*
17941	and flush* cache after writing to mapped pointer.*
17942	Map/unmap operations don't do that automatically.
17943	Vulkan provides following functions for this purpose `vkFlushMappedMemoryRanges()`,
17944	`vkInvalidateMappedMemoryRanges()`, but this library provides more convenient
17945	functions that refer to given allocation object: vmaFlushAllocation(),
17946	vmaInvalidateAllocation(),
17947	or multiple objects at once: vmaFlushAllocations(), vmaInvalidateAllocations().
17948
17949	Regions of memory specified for flush/invalidate must be aligned to
17950	`VkPhysicalDeviceLimits::nonCoherentAtomSize`. This is automatically ensured by the library.
17951	In any memory type that is `HOST_VISIBLE` but not `HOST_COHERENT`, all allocations
17952	within blocks are aligned to this value, so their offsets are always multiply of
17953	`nonCoherentAtomSize` and two different allocations never share same "line" of this size.
17954
17955	Also, Windows drivers from all 3 PC GPU vendors (AMD, Intel, NVIDIA)
17956	currently provide `HOST_COHERENT` flag on all memory types that are
17957	`HOST_VISIBLE`, so on PC you may not need to bother.
17958
17959
17960	\page staying_within_budget Staying within budget
17961
17962	When developing a graphics-intensive game or program, it is important to avoid allocating
17963	more GPU memory than it is physically available. When the memory is over-committed,
17964	various bad things can happen, depending on the specific GPU, graphics driver, and
17965	operating system:
17966
17967	- It may just work without any problems.
17968	- The application may slow down because some memory blocks are moved to system RAM
17969	and the GPU has to access them through PCI Express bus.
17970	- A new allocation may take very long time to complete, even few seconds, and possibly
17971	freeze entire system.
17972	- The new allocation may fail with `VK_ERROR_OUT_OF_DEVICE_MEMORY`.
17973	- It may even result in GPU crash (TDR), observed as `VK_ERROR_DEVICE_LOST`
17974	returned somewhere later.
17975
17976	\section staying_within_budget_querying_for_budget Querying for budget
17977
17978	To query for current memory usage and available budget, use function vmaGetHeapBudgets().
17979	Returned structure #VmaBudget contains quantities expressed in bytes, per Vulkan memory heap.
17980
17981	Please note that this function returns different information and works faster than
17982	vmaCalculateStatistics(). vmaGetHeapBudgets() can be called every frame or even before every
17983	allocation, while vmaCalculateStatistics() is intended to be used rarely,
17984	only to obtain statistical information, e.g. for debugging purposes.
17985
17986	It is recommended to use <b>VK_EXT_memory_budget</b> device extension to obtain information
17987	about the budget from Vulkan device. VMA is able to use this extension automatically.
17988	When not enabled, the allocator behaves same way, but then it estimates current usage
17989	and available budget based on its internal information and Vulkan memory heap sizes,
17990	which may be less precise. In order to use this extension:
17991
17992	1. Make sure extensions VK_EXT_memory_budget and VK_KHR_get_physical_device_properties2
17993	required by it are available and enable them. Please note that the first is a device
17994	extension and the second is instance extension!
17995	2. Use flag #VMA_ALLOCATOR_CREATE_EXT_MEMORY_BUDGET_BIT when creating #VmaAllocator object.
17996	3. Make sure to call vmaSetCurrentFrameIndex() every frame. Budget is queried from
17997	Vulkan inside of it to avoid overhead of querying it with every allocation.
17998
17999	\section staying_within_budget_controlling_memory_usage Controlling memory usage
18000
18001	There are many ways in which you can try to stay within the budget.
18002
18003	First, when making new allocation requires allocating a new memory block, the library
18004	tries not to exceed the budget automatically. If a block with default recommended size
18005	(e.g. 256 MB) would go over budget, a smaller block is allocated, possibly even
18006	dedicated memory for just this resource.
18007
18008	If the size of the requested resource plus current memory usage is more than the
18009	budget, by default the library still tries to create it, leaving it to the Vulkan
18010	implementation whether the allocation succeeds or fails. You can change this behavior
18011	by using #VMA_ALLOCATION_CREATE_WITHIN_BUDGET_BIT flag. With it, the allocation is
18012	not made if it would exceed the budget or if the budget is already exceeded.
18013	VMA then tries to make the allocation from the next eligible Vulkan memory type.
18014	The all of them fail, the call then fails with `VK_ERROR_OUT_OF_DEVICE_MEMORY`.
18015	Example usage pattern may be to pass the #VMA_ALLOCATION_CREATE_WITHIN_BUDGET_BIT flag
18016	when creating resources that are not essential for the application (e.g. the texture
18017	of a specific object) and not to pass it when creating critically important resources
18018	(e.g. render targets).
18019
18020	On AMD graphics cards there is a custom vendor extension available: <b>VK_AMD_memory_overallocation_behavior</b>
18021	that allows to control the behavior of the Vulkan implementation in out-of-memory cases -
18022	whether it should fail with an error code or still allow the allocation.
18023	Usage of this extension involves only passing extra structure on Vulkan device creation,
18024	so it is out of scope of this library.
18025
18026	Finally, you can also use #VMA_ALLOCATION_CREATE_NEVER_ALLOCATE_BIT flag to make sure
18027	a new allocation is created only when it fits inside one of the existing memory blocks.
18028	If it would require to allocate a new block, if fails instead with `VK_ERROR_OUT_OF_DEVICE_MEMORY`.
18029	This also ensures that the function call is very fast because it never goes to Vulkan
18030	to obtain a new block.
18031
18032	\note Creating \ref custom_memory_pools with VmaPoolCreateInfo::minBlockCount
18033	set to more than 0 will currently try to allocate memory blocks without checking whether they
18034	fit within budget.
18035
18036
18037	\page resource_aliasing Resource aliasing (overlap)
18038
18039	New explicit graphics APIs (Vulkan and Direct3D 12), thanks to manual memory
18040	management, give an opportunity to alias (overlap) multiple resources in the
18041	same region of memory - a feature not available in the old APIs (Direct3D 11, OpenGL).
18042	It can be useful to save video memory, but it must be used with caution.
18043
18044	For example, if you know the flow of your whole render frame in advance, you
18045	are going to use some intermediate textures or buffers only during a small range of render passes,
18046	and you know these ranges don't overlap in time, you can bind these resources to
18047	the same place in memory, even if they have completely different parameters (width, height, format etc.).
18048
18049	![Resource aliasing (overlap)](../gfx/Aliasing.png)
18050
18051	Such scenario is possible using VMA, but you need to create your images manually.
18052	Then you need to calculate parameters of an allocation to be made using formula:
18053
18054	- allocation size = max(size of each image)
18055	- allocation alignment = max(alignment of each image)
18056	- allocation memoryTypeBits = bitwise AND(memoryTypeBits of each image)
18057
18058	Following example shows two different images bound to the same place in memory,
18059	allocated to fit largest of them.
18060
18061	\code
18062	// A 512x512 texture to be sampled.
18063	VkImageCreateInfo img1CreateInfo = { VK_STRUCTURE_TYPE_IMAGE_CREATE_INFO };
18064	img1CreateInfo.imageType = VK_IMAGE_TYPE_2D;
18065	img1CreateInfo.extent.width = 512;
18066	img1CreateInfo.extent.height = 512;
18067	img1CreateInfo.extent.depth = 1;
18068	img1CreateInfo.mipLevels = 10;
18069	img1CreateInfo.arrayLayers = 1;
18070	img1CreateInfo.format = VK_FORMAT_R8G8B8A8_SRGB;
18071	img1CreateInfo.tiling = VK_IMAGE_TILING_OPTIMAL;
18072	img1CreateInfo.initialLayout = VK_IMAGE_LAYOUT_UNDEFINED;
18073	img1CreateInfo.usage = VK_IMAGE_USAGE_TRANSFER_DST_BIT \| VK_IMAGE_USAGE_SAMPLED_BIT;
18074	img1CreateInfo.samples = VK_SAMPLE_COUNT_1_BIT;
18075
18076	// A full screen texture to be used as color attachment.
18077	VkImageCreateInfo img2CreateInfo = { VK_STRUCTURE_TYPE_IMAGE_CREATE_INFO };
18078	img2CreateInfo.imageType = VK_IMAGE_TYPE_2D;
18079	img2CreateInfo.extent.width = 1920;
18080	img2CreateInfo.extent.height = 1080;
18081	img2CreateInfo.extent.depth = 1;
18082	img2CreateInfo.mipLevels = 1;
18083	img2CreateInfo.arrayLayers = 1;
18084	img2CreateInfo.format = VK_FORMAT_R8G8B8A8_UNORM;
18085	img2CreateInfo.tiling = VK_IMAGE_TILING_OPTIMAL;
18086	img2CreateInfo.initialLayout = VK_IMAGE_LAYOUT_UNDEFINED;
18087	img2CreateInfo.usage = VK_IMAGE_USAGE_SAMPLED_BIT \| VK_IMAGE_USAGE_COLOR_ATTACHMENT_BIT;
18088	img2CreateInfo.samples = VK_SAMPLE_COUNT_1_BIT;
18089
18090	VkImage img1;
18091	res = vkCreateImage(device, &img1CreateInfo, nullptr, &img1);
18092	VkImage img2;
18093	res = vkCreateImage(device, &img2CreateInfo, nullptr, &img2);
18094
18095	VkMemoryRequirements img1MemReq;
18096	vkGetImageMemoryRequirements(device, img1, &img1MemReq);
18097	VkMemoryRequirements img2MemReq;
18098	vkGetImageMemoryRequirements(device, img2, &img2MemReq);
18099
18100	VkMemoryRequirements finalMemReq = {};
18101	finalMemReq.size = std::max(img1MemReq.size, img2MemReq.size);
18102	finalMemReq.alignment = std::max(img1MemReq.alignment, img2MemReq.alignment);
18103	finalMemReq.memoryTypeBits = img1MemReq.memoryTypeBits & img2MemReq.memoryTypeBits;
18104	// Validate if(finalMemReq.memoryTypeBits != 0)
18105
18106	VmaAllocationCreateInfo allocCreateInfo = {};
18107	allocCreateInfo.preferredFlags = VK_MEMORY_PROPERTY_DEVICE_LOCAL_BIT;
18108
18109	VmaAllocation alloc;
18110	res = vmaAllocateMemory(allocator, &finalMemReq, &allocCreateInfo, &alloc, nullptr);
18111
18112	res = vmaBindImageMemory(allocator, alloc, img1);
18113	res = vmaBindImageMemory(allocator, alloc, img2);
18114
18115	// You can use img1, img2 here, but not at the same time!
18116
18117	vmaFreeMemory(allocator, alloc);
18118	vkDestroyImage(allocator, img2, nullptr);
18119	vkDestroyImage(allocator, img1, nullptr);
18120	\endcode
18121
18122	Remember that using resources that alias in memory requires proper synchronization.
18123	You need to issue a memory barrier to make sure commands that use `img1` and `img2`
18124	don't overlap on GPU timeline.
18125	You also need to treat a resource after aliasing as uninitialized - containing garbage data.
18126	For example, if you use `img1` and then want to use `img2`, you need to issue
18127	an image memory barrier for `img2` with `oldLayout` = `VK_IMAGE_LAYOUT_UNDEFINED`.
18128
18129	Additional considerations:
18130
18131	- Vulkan also allows to interpret contents of memory between aliasing resources consistently in some cases.
18132	See chapter 11.8. "Memory Aliasing" of Vulkan specification or `VK_IMAGE_CREATE_ALIAS_BIT` flag.
18133	- You can create more complex layout where different images and buffers are bound
18134	at different offsets inside one large allocation. For example, one can imagine
18135	a big texture used in some render passes, aliasing with a set of many small buffers
18136	used between in some further passes. To bind a resource at non-zero offset in an allocation,
18137	use vmaBindBufferMemory2() / vmaBindImageMemory2().
18138	- Before allocating memory for the resources you want to alias, check `memoryTypeBits`
18139	returned in memory requirements of each resource to make sure the bits overlap.
18140	Some GPUs may expose multiple memory types suitable e.g. only for buffers or
18141	images with `COLOR_ATTACHMENT` usage, so the sets of memory types supported by your
18142	resources may be disjoint. Aliasing them is not possible in that case.
18143
18144
18145	\page custom_memory_pools Custom memory pools
18146
18147	A memory pool contains a number of `VkDeviceMemory` blocks.
18148	The library automatically creates and manages default pool for each memory type available on the device.
18149	Default memory pool automatically grows in size.
18150	Size of allocated blocks is also variable and managed automatically.
18151
18152	You can create custom pool and allocate memory out of it.
18153	It can be useful if you want to:
18154
18155	- Keep certain kind of allocations separate from others.
18156	- Enforce particular, fixed size of Vulkan memory blocks.
18157	- Limit maximum amount of Vulkan memory allocated for that pool.
18158	- Reserve minimum or fixed amount of Vulkan memory always preallocated for that pool.
18159	- Use extra parameters for a set of your allocations that are available in #VmaPoolCreateInfo but not in
18160	#VmaAllocationCreateInfo - e.g., custom minimum alignment, custom `pNext` chain.
18161	- Perform defragmentation on a specific subset of your allocations.
18162
18163	To use custom memory pools:
18164
18165	-# Fill VmaPoolCreateInfo structure.
18166	-# Call vmaCreatePool() to obtain #VmaPool handle.
18167	-# When making an allocation, set VmaAllocationCreateInfo::pool to this handle.
18168	You don't need to specify any other parameters of this structure, like `usage`.
18169
18170	Example:
18171
18172	\code
18173	// Find memoryTypeIndex for the pool.
18174	VkBufferCreateInfo sampleBufCreateInfo = { VK_STRUCTURE_TYPE_BUFFER_CREATE_INFO };
18175	sampleBufCreateInfo.size = 0x10000; // Doesn't matter.
18176	sampleBufCreateInfo.usage = VK_BUFFER_USAGE_UNIFORM_BUFFER_BIT \| VK_BUFFER_USAGE_TRANSFER_DST_BIT;
18177
18178	VmaAllocationCreateInfo sampleAllocCreateInfo = {};
18179	sampleAllocCreateInfo.usage = VMA_MEMORY_USAGE_AUTO;
18180
18181	uint32_t memTypeIndex;
18182	VkResult res = vmaFindMemoryTypeIndexForBufferInfo(allocator,
18183	&sampleBufCreateInfo, &sampleAllocCreateInfo, &memTypeIndex);
18184	// Check res...
18185
18186	// Create a pool that can have at most 2 blocks, 128 MiB each.
18187	VmaPoolCreateInfo poolCreateInfo = {};
18188	poolCreateInfo.memoryTypeIndex = memTypeIndex;
18189	poolCreateInfo.blockSize = 128ull 1024 * 1024;*
18190	poolCreateInfo.maxBlockCount = 2;
18191
18192	VmaPool pool;
18193	res = vmaCreatePool(allocator, &poolCreateInfo, &pool);
18194	// Check res...
18195
18196	// Allocate a buffer out of it.
18197	VkBufferCreateInfo bufCreateInfo = { VK_STRUCTURE_TYPE_BUFFER_CREATE_INFO };
18198	bufCreateInfo.size = 1024;
18199	bufCreateInfo.usage = VK_BUFFER_USAGE_UNIFORM_BUFFER_BIT \| VK_BUFFER_USAGE_TRANSFER_DST_BIT;
18200
18201	VmaAllocationCreateInfo allocCreateInfo = {};
18202	allocCreateInfo.pool = pool;
18203
18204	VkBuffer buf;
18205	VmaAllocation alloc;
18206	res = vmaCreateBuffer(allocator, &bufCreateInfo, &allocCreateInfo, &buf, &alloc, nullptr);
18207	// Check res...
18208	\endcode
18209
18210	You have to free all allocations made from this pool before destroying it.
18211
18212	\code
18213	vmaDestroyBuffer(allocator, buf, alloc);
18214	vmaDestroyPool(allocator, pool);
18215	\endcode
18216
18217	New versions of this library support creating dedicated allocations in custom pools.
18218	It is supported only when VmaPoolCreateInfo::blockSize = 0.
18219	To use this feature, set VmaAllocationCreateInfo::pool to the pointer to your custom pool and
18220	VmaAllocationCreateInfo::flags to #VMA_ALLOCATION_CREATE_DEDICATED_MEMORY_BIT.
18221
18222	\note Excessive use of custom pools is a common mistake when using this library.
18223	Custom pools may be useful for special purposes - when you want to
18224	keep certain type of resources separate e.g. to reserve minimum amount of memory
18225	for them or limit maximum amount of memory they can occupy. For most
18226	resources this is not needed and so it is not recommended to create #VmaPool
18227	objects and allocations out of them. Allocating from the default pool is sufficient.
18228
18229
18230	\section custom_memory_pools_MemTypeIndex Choosing memory type index
18231
18232	When creating a pool, you must explicitly specify memory type index.
18233	To find the one suitable for your buffers or images, you can use helper functions
18234	vmaFindMemoryTypeIndexForBufferInfo(), vmaFindMemoryTypeIndexForImageInfo().
18235	You need to provide structures with example parameters of buffers or images
18236	that you are going to create in that pool.
18237
18238	\code
18239	VkBufferCreateInfo exampleBufCreateInfo = { VK_STRUCTURE_TYPE_BUFFER_CREATE_INFO };
18240	exampleBufCreateInfo.size = 1024; // Doesn't matter
18241	exampleBufCreateInfo.usage = VK_BUFFER_USAGE_UNIFORM_BUFFER_BIT \| VK_BUFFER_USAGE_TRANSFER_DST_BIT;
18242
18243	VmaAllocationCreateInfo allocCreateInfo = {};
18244	allocCreateInfo.usage = VMA_MEMORY_USAGE_AUTO;
18245
18246	uint32_t memTypeIndex;
18247	vmaFindMemoryTypeIndexForBufferInfo(allocator, &exampleBufCreateInfo, &allocCreateInfo, &memTypeIndex);
18248
18249	VmaPoolCreateInfo poolCreateInfo = {};
18250	poolCreateInfo.memoryTypeIndex = memTypeIndex;
18251	// ...
18252	\endcode
18253
18254	When creating buffers/images allocated in that pool, provide following parameters:
18255
18256	- `VkBufferCreateInfo`: Prefer to pass same parameters as above.
18257	Otherwise you risk creating resources in a memory type that is not suitable for them, which may result in undefined behavior.
18258	Using different `VK_BUFFER_USAGE_` flags may work, but you shouldn't create images in a pool intended for buffers
18259	or the other way around.
18260	- VmaAllocationCreateInfo: You don't need to pass same parameters. Fill only `pool` member.
18261	Other members are ignored anyway.
18262
18263	\section linear_algorithm Linear allocation algorithm
18264
18265	Each Vulkan memory block managed by this library has accompanying metadata that
18266	keeps track of used and unused regions. By default, the metadata structure and
18267	algorithm tries to find best place for new allocations among free regions to
18268	optimize memory usage. This way you can allocate and free objects in any order.
18269
18270	![Default allocation algorithm](../gfx/Linear_allocator_1_algo_default.png)
18271
18272	Sometimes there is a need to use simpler, linear allocation algorithm. You can
18273	create custom pool that uses such algorithm by adding flag
18274	#VMA_POOL_CREATE_LINEAR_ALGORITHM_BIT to VmaPoolCreateInfo::flags while creating
18275	#VmaPool object. Then an alternative metadata management is used. It always
18276	creates new allocations after last one and doesn't reuse free regions after
18277	allocations freed in the middle. It results in better allocation performance and
18278	less memory consumed by metadata.
18279
18280	![Linear allocation algorithm](../gfx/Linear_allocator_2_algo_linear.png)
18281
18282	With this one flag, you can create a custom pool that can be used in many ways:
18283	free-at-once, stack, double stack, and ring buffer. See below for details.
18284	You don't need to specify explicitly which of these options you are going to use - it is detected automatically.
18285
18286	\subsection linear_algorithm_free_at_once Free-at-once
18287
18288	In a pool that uses linear algorithm, you still need to free all the allocations
18289	individually, e.g. by using vmaFreeMemory() or vmaDestroyBuffer(). You can free
18290	them in any order. New allocations are always made after last one - free space
18291	in the middle is not reused. However, when you release all the allocation and
18292	the pool becomes empty, allocation starts from the beginning again. This way you
18293	can use linear algorithm to speed up creation of allocations that you are going
18294	to release all at once.
18295
18296	![Free-at-once](../gfx/Linear_allocator_3_free_at_once.png)
18297
18298	This mode is also available for pools created with VmaPoolCreateInfo::maxBlockCount
18299	value that allows multiple memory blocks.
18300
18301	\subsection linear_algorithm_stack Stack
18302
18303	When you free an allocation that was created last, its space can be reused.
18304	Thanks to this, if you always release allocations in the order opposite to their
18305	creation (LIFO - Last In First Out), you can achieve behavior of a stack.
18306
18307	![Stack](../gfx/Linear_allocator_4_stack.png)
18308
18309	This mode is also available for pools created with VmaPoolCreateInfo::maxBlockCount
18310	value that allows multiple memory blocks.
18311
18312	\subsection linear_algorithm_double_stack Double stack
18313
18314	The space reserved by a custom pool with linear algorithm may be used by two
18315	stacks:
18316
18317	- First, default one, growing up from offset 0.
18318	- Second, "upper" one, growing down from the end towards lower offsets.
18319
18320	To make allocation from the upper stack, add flag #VMA_ALLOCATION_CREATE_UPPER_ADDRESS_BIT
18321	to VmaAllocationCreateInfo::flags.
18322
18323	![Double stack](../gfx/Linear_allocator_7_double_stack.png)
18324
18325	Double stack is available only in pools with one memory block -
18326	VmaPoolCreateInfo::maxBlockCount must be 1. Otherwise behavior is undefined.
18327
18328	When the two stacks' ends meet so there is not enough space between them for a
18329	new allocation, such allocation fails with usual
18330	`VK_ERROR_OUT_OF_DEVICE_MEMORY` error.
18331
18332	\subsection linear_algorithm_ring_buffer Ring buffer
18333
18334	When you free some allocations from the beginning and there is not enough free space
18335	for a new one at the end of a pool, allocator's "cursor" wraps around to the
18336	beginning and starts allocation there. Thanks to this, if you always release
18337	allocations in the same order as you created them (FIFO - First In First Out),
18338	you can achieve behavior of a ring buffer / queue.
18339
18340	![Ring buffer](../gfx/Linear_allocator_5_ring_buffer.png)
18341
18342	Ring buffer is available only in pools with one memory block -
18343	VmaPoolCreateInfo::maxBlockCount must be 1. Otherwise behavior is undefined.
18344
18345	\note \ref defragmentation is not supported in custom pools created with #VMA_POOL_CREATE_LINEAR_ALGORITHM_BIT.
18346
18347
18348	\page defragmentation Defragmentation
18349
18350	Interleaved allocations and deallocations of many objects of varying size can
18351	cause fragmentation over time, which can lead to a situation where the library is unable
18352	to find a continuous range of free memory for a new allocation despite there is
18353	enough free space, just scattered across many small free ranges between existing
18354	allocations.
18355
18356	To mitigate this problem, you can use defragmentation feature.
18357	It doesn't happen automatically though and needs your cooperation,
18358	because VMA is a low level library that only allocates memory.
18359	It cannot recreate buffers and images in a new place as it doesn't remember the contents of `VkBufferCreateInfo` / `VkImageCreateInfo` structures.
18360	It cannot copy their contents as it doesn't record any commands to a command buffer.
18361
18362	Example:
18363
18364	\code
18365	VmaDefragmentationInfo defragInfo = {};
18366	defragInfo.pool = myPool;
18367	defragInfo.flags = VMA_DEFRAGMENTATION_FLAG_ALGORITHM_FAST_BIT;
18368
18369	VmaDefragmentationContext defragCtx;
18370	VkResult res = vmaBeginDefragmentation(allocator, &defragInfo, &defragCtx);
18371	// Check res...
18372
18373	for(;;)
18374	{
18375	VmaDefragmentationPassMoveInfo pass;
18376	res = vmaBeginDefragmentationPass(allocator, defragCtx, &pass);
18377	if(res == VK_SUCCESS)
18378	break;
18379	else if(res != VK_INCOMPLETE)
18380	// Handle error...
18381
18382	for(uint32_t i = 0; i < pass.moveCount; ++i)
18383	{
18384	// Inspect pass.pMoves[i].srcAllocation, identify what buffer/image it represents.
18385	VmaAllocationInfo allocInfo;
18386	vmaGetAllocationInfo(allocator, pMoves[i].srcAllocation, &allocInfo);
18387	MyEngineResourceData resData = (MyEngineResourceData)allocInfo.pUserData;
18388
18389	// Recreate and bind this buffer/image at: pass.pMoves[i].dstMemory, pass.pMoves[i].dstOffset.
18390	VkImageCreateInfo imgCreateInfo = ...
18391	VkImage newImg;
18392	res = vkCreateImage(device, &imgCreateInfo, nullptr, &newImg);
18393	// Check res...
18394	res = vmaBindImageMemory(allocator, pMoves[i].dstTmpAllocation, newImg);
18395	// Check res...
18396
18397	// Issue a vkCmdCopyBuffer/vkCmdCopyImage to copy its content to the new place.
18398	vkCmdCopyImage(cmdBuf, resData->img, ..., newImg, ...);
18399	}
18400
18401	// Make sure the copy commands finished executing.
18402	vkWaitForFences(...);
18403
18404	// Destroy old buffers/images bound with pass.pMoves[i].srcAllocation.
18405	for(uint32_t i = 0; i < pass.moveCount; ++i)
18406	{
18407	// ...
18408	vkDestroyImage(device, resData->img, nullptr);
18409	}
18410
18411	// Update appropriate descriptors to point to the new places...
18412
18413	res = vmaEndDefragmentationPass(allocator, defragCtx, &pass);
18414	if(res == VK_SUCCESS)
18415	break;
18416	else if(res != VK_INCOMPLETE)
18417	// Handle error...
18418	}
18419
18420	vmaEndDefragmentation(allocator, defragCtx, nullptr);
18421	\endcode
18422
18423	Although functions like vmaCreateBuffer(), vmaCreateImage(), vmaDestroyBuffer(), vmaDestroyImage()
18424	create/destroy an allocation and a buffer/image at once, these are just a shortcut for
18425	creating the resource, allocating memory, and binding them together.
18426	Defragmentation works on memory allocations only. You must handle the rest manually.
18427	Defragmentation is an iterative process that should repreat "passes" as long as related functions
18428	return `VK_INCOMPLETE` not `VK_SUCCESS`.
18429	In each pass:
18430
18431	1. vmaBeginDefragmentationPass() function call:
18432	- Calculates and returns the list of allocations to be moved in this pass.
18433	Note this can be a time-consuming process.
18434	- Reserves destination memory for them by creating temporary destination allocations
18435	that you can query for their `VkDeviceMemory` + offset using vmaGetAllocationInfo().
18436	2. Inside the pass, you should:
18437	- Inspect the returned list of allocations to be moved.
18438	- Create new buffers/images and bind them at the returned destination temporary allocations.
18439	- Copy data from source to destination resources if necessary.
18440	- Destroy the source buffers/images, but NOT their allocations.
18441	3. vmaEndDefragmentationPass() function call:
18442	- Frees the source memory reserved for the allocations that are moved.
18443	- Modifies source #VmaAllocation objects that are moved to point to the destination reserved memory.
18444	- Frees `VkDeviceMemory` blocks that became empty.
18445
18446	Unlike in previous iterations of the defragmentation API, there is no list of "movable" allocations passed as a parameter.
18447	Defragmentation algorithm tries to move all suitable allocations.
18448	You can, however, refuse to move some of them inside a defragmentation pass, by setting
18449	`pass.pMoves[i].operation` to #VMA_DEFRAGMENTATION_MOVE_OPERATION_IGNORE.
18450	This is not recommended and may result in suboptimal packing of the allocations after defragmentation.
18451	If you cannot ensure any allocation can be moved, it is better to keep movable allocations separate in a custom pool.
18452
18453	Inside a pass, for each allocation that should be moved:
18454
18455	- You should copy its data from the source to the destination place by calling e.g. `vkCmdCopyBuffer()`, `vkCmdCopyImage()`.
18456	- You need to make sure these commands finished executing before destroying the source buffers/images and before calling vmaEndDefragmentationPass().
18457	- If a resource doesn't contain any meaningful data, e.g. it is a transient color attachment image to be cleared,
18458	filled, and used temporarily in each rendering frame, you can just recreate this image
18459	without copying its data.
18460	- If the resource is in `HOST_VISIBLE` and `HOST_CACHED` memory, you can copy its data on the CPU
18461	using `memcpy()`.
18462	- If you cannot move the allocation, you can set `pass.pMoves[i].operation` to #VMA_DEFRAGMENTATION_MOVE_OPERATION_IGNORE.
18463	This will cancel the move.
18464	- vmaEndDefragmentationPass() will then free the destination memory
18465	not the source memory of the allocation, leaving it unchanged.
18466	- If you decide the allocation is unimportant and can be destroyed instead of moved (e.g. it wasn't used for long time),
18467	you can set `pass.pMoves[i].operation` to #VMA_DEFRAGMENTATION_MOVE_OPERATION_DESTROY.
18468	- vmaEndDefragmentationPass() will then free both source and destination memory, and will destroy the source #VmaAllocation object.
18469
18470	You can defragment a specific custom pool by setting VmaDefragmentationInfo::pool
18471	(like in the example above) or all the default pools by setting this member to null.
18472
18473	Defragmentation is always performed in each pool separately.
18474	Allocations are never moved between different Vulkan memory types.
18475	The size of the destination memory reserved for a moved allocation is the same as the original one.
18476	Alignment of an allocation as it was determined using `vkGetBufferMemoryRequirements()` etc. is also respected after defragmentation.
18477	Buffers/images should be recreated with the same `VkBufferCreateInfo` / `VkImageCreateInfo` parameters as the original ones.
18478
18479	You can perform the defragmentation incrementally to limit the number of allocations and bytes to be moved
18480	in each pass, e.g. to call it in sync with render frames and not to experience too big hitches.
18481	See members: VmaDefragmentationInfo::maxBytesPerPass, VmaDefragmentationInfo::maxAllocationsPerPass.
18482
18483	It is also safe to perform the defragmentation asynchronously to render frames and other Vulkan and VMA
18484	usage, possibly from multiple threads, with the exception that allocations
18485	returned in VmaDefragmentationPassMoveInfo::pMoves shouldn't be destroyed until the defragmentation pass is ended.
18486
18487	<b>Mapping</b> is preserved on allocations that are moved during defragmentation.
18488	Whether through #VMA_ALLOCATION_CREATE_MAPPED_BIT or vmaMapMemory(), the allocations
18489	are mapped at their new place. Of course, pointer to the mapped data changes, so it needs to be queried
18490	using VmaAllocationInfo::pMappedData.
18491
18492	\note Defragmentation is not supported in custom pools created with #VMA_POOL_CREATE_LINEAR_ALGORITHM_BIT.
18493
18494
18495	\page statistics Statistics
18496
18497	This library contains several functions that return information about its internal state,
18498	especially the amount of memory allocated from Vulkan.
18499
18500	\section statistics_numeric_statistics Numeric statistics
18501
18502	If you need to obtain basic statistics about memory usage per heap, together with current budget,
18503	you can call function vmaGetHeapBudgets() and inspect structure #VmaBudget.
18504	This is useful to keep track of memory usage and stay withing budget
18505	(see also \ref staying_within_budget).
18506	Example:
18507
18508	\code
18509	uint32_t heapIndex = ...
18510
18511	VmaBudget budgets[VK_MAX_MEMORY_HEAPS];
18512	vmaGetHeapBudgets(allocator, budgets);
18513
18514	printf("My heap currently has %u allocations taking %llu B,\n",
18515	budgets[heapIndex].statistics.allocationCount,
18516	budgets[heapIndex].statistics.allocationBytes);
18517	printf("allocated out of %u Vulkan device memory blocks taking %llu B,\n",
18518	budgets[heapIndex].statistics.blockCount,
18519	budgets[heapIndex].statistics.blockBytes);
18520	printf("Vulkan reports total usage %llu B with budget %llu B.\n",
18521	budgets[heapIndex].usage,
18522	budgets[heapIndex].budget);
18523	\endcode
18524
18525	You can query for more detailed statistics per memory heap, type, and totals,
18526	including minimum and maximum allocation size and unused range size,
18527	by calling function vmaCalculateStatistics() and inspecting structure #VmaTotalStatistics.
18528	This function is slower though, as it has to traverse all the internal data structures,
18529	so it should be used only for debugging purposes.
18530
18531	You can query for statistics of a custom pool using function vmaGetPoolStatistics()
18532	or vmaCalculatePoolStatistics().
18533
18534	You can query for information about a specific allocation using function vmaGetAllocationInfo().
18535	It fill structure #VmaAllocationInfo.
18536
18537	\section statistics_json_dump JSON dump
18538
18539	You can dump internal state of the allocator to a string in JSON format using function vmaBuildStatsString().
18540	The result is guaranteed to be correct JSON.
18541	It uses ANSI encoding.
18542	Any strings provided by user (see [Allocation names](@ref allocation_names))
18543	are copied as-is and properly escaped for JSON, so if they use UTF-8, ISO-8859-2 or any other encoding,
18544	this JSON string can be treated as using this encoding.
18545	It must be freed using function vmaFreeStatsString().
18546
18547	The format of this JSON string is not part of official documentation of the library,
18548	but it will not change in backward-incompatible way without increasing library major version number
18549	and appropriate mention in changelog.
18550
18551	The JSON string contains all the data that can be obtained using vmaCalculateStatistics().
18552	It can also contain detailed map of allocated memory blocks and their regions -
18553	free and occupied by allocations.
18554	This allows e.g. to visualize the memory or assess fragmentation.
18555
18556
18557	\page allocation_annotation Allocation names and user data
18558
18559	\section allocation_user_data Allocation user data
18560
18561	You can annotate allocations with your own information, e.g. for debugging purposes.
18562	To do that, fill VmaAllocationCreateInfo::pUserData field when creating
18563	an allocation. It is an opaque `void` pointer. You can use it e.g. as a pointer,*
18564	some handle, index, key, ordinal number or any other value that would associate
18565	the allocation with your custom metadata.
18566	It is useful to identify appropriate data structures in your engine given #VmaAllocation,
18567	e.g. when doing \ref defragmentation.
18568
18569	\code
18570	VkBufferCreateInfo bufCreateInfo = ...
18571
18572	MyBufferMetadata pMetadata = CreateBufferMetadata();*
18573
18574	VmaAllocationCreateInfo allocCreateInfo = {};
18575	allocCreateInfo.usage = VMA_MEMORY_USAGE_AUTO;
18576	allocCreateInfo.pUserData = pMetadata;
18577
18578	VkBuffer buffer;
18579	VmaAllocation allocation;
18580	vmaCreateBuffer(allocator, &bufCreateInfo, &allocCreateInfo, &buffer, &allocation, nullptr);
18581	\endcode
18582
18583	The pointer may be later retrieved as VmaAllocationInfo::pUserData:
18584
18585	\code
18586	VmaAllocationInfo allocInfo;
18587	vmaGetAllocationInfo(allocator, allocation, &allocInfo);
18588	MyBufferMetadata pMetadata = (MyBufferMetadata)allocInfo.pUserData;
18589	\endcode
18590
18591	It can also be changed using function vmaSetAllocationUserData().
18592
18593	Values of (non-zero) allocations' `pUserData` are printed in JSON report created by
18594	vmaBuildStatsString() in hexadecimal form.
18595
18596	\section allocation_names Allocation names
18597
18598	An allocation can also carry a null-terminated string, giving a name to the allocation.
18599	To set it, call vmaSetAllocationName().
18600	The library creates internal copy of the string, so the pointer you pass doesn't need
18601	to be valid for whole lifetime of the allocation. You can free it after the call.
18602
18603	\code
18604	std::string imageName = "Texture: ";
18605	imageName += fileName;
18606	vmaSetAllocationName(allocator, allocation, imageName.c_str());
18607	\endcode
18608
18609	The string can be later retrieved by inspecting VmaAllocationInfo::pName.
18610	It is also printed in JSON report created by vmaBuildStatsString().
18611
18612	\note Setting string name to VMA allocation doesn't automatically set it to the Vulkan buffer or image created with it.
18613	You must do it manually using an extension like VK_EXT_debug_utils, which is independent of this library.
18614
18615
18616	\page virtual_allocator Virtual allocator
18617
18618	As an extra feature, the core allocation algorithm of the library is exposed through a simple and convenient API of "virtual allocator".
18619	It doesn't allocate any real GPU memory. It just keeps track of used and free regions of a "virtual block".
18620	You can use it to allocate your own memory or other objects, even completely unrelated to Vulkan.
18621	A common use case is sub-allocation of pieces of one large GPU buffer.
18622
18623	\section virtual_allocator_creating_virtual_block Creating virtual block
18624
18625	To use this functionality, there is no main "allocator" object.
18626	You don't need to have #VmaAllocator object created.
18627	All you need to do is to create a separate #VmaVirtualBlock object for each block of memory you want to be managed by the allocator:
18628
18629	-# Fill in #VmaVirtualBlockCreateInfo structure.
18630	-# Call vmaCreateVirtualBlock(). Get new #VmaVirtualBlock object.
18631
18632	Example:
18633
18634	\code
18635	VmaVirtualBlockCreateInfo blockCreateInfo = {};
18636	blockCreateInfo.size = 1048576; // 1 MB
18637
18638	VmaVirtualBlock block;
18639	VkResult res = vmaCreateVirtualBlock(&blockCreateInfo, &block);
18640	\endcode
18641
18642	\section virtual_allocator_making_virtual_allocations Making virtual allocations
18643
18644	#VmaVirtualBlock object contains internal data structure that keeps track of free and occupied regions
18645	using the same code as the main Vulkan memory allocator.
18646	Similarly to #VmaAllocation for standard GPU allocations, there is #VmaVirtualAllocation type
18647	that represents an opaque handle to an allocation withing the virtual block.
18648
18649	In order to make such allocation:
18650
18651	-# Fill in #VmaVirtualAllocationCreateInfo structure.
18652	-# Call vmaVirtualAllocate(). Get new #VmaVirtualAllocation object that represents the allocation.
18653	You can also receive `VkDeviceSize offset` that was assigned to the allocation.
18654
18655	Example:
18656
18657	\code
18658	VmaVirtualAllocationCreateInfo allocCreateInfo = {};
18659	allocCreateInfo.size = 4096; // 4 KB
18660
18661	VmaVirtualAllocation alloc;
18662	VkDeviceSize offset;
18663	res = vmaVirtualAllocate(block, &allocCreateInfo, &alloc, &offset);
18664	if(res == VK_SUCCESS)
18665	{
18666	// Use the 4 KB of your memory starting at offset.
18667	}
18668	else
18669	{
18670	// Allocation failed - no space for it could be found. Handle this error!
18671	}
18672	\endcode
18673
18674	\section virtual_allocator_deallocation Deallocation
18675
18676	When no longer needed, an allocation can be freed by calling vmaVirtualFree().
18677	You can only pass to this function an allocation that was previously returned by vmaVirtualAllocate()
18678	called for the same #VmaVirtualBlock.
18679
18680	When whole block is no longer needed, the block object can be released by calling vmaDestroyVirtualBlock().
18681	All allocations must be freed before the block is destroyed, which is checked internally by an assert.
18682	However, if you don't want to call vmaVirtualFree() for each allocation, you can use vmaClearVirtualBlock() to free them all at once -
18683	a feature not available in normal Vulkan memory allocator. Example:
18684
18685	\code
18686	vmaVirtualFree(block, alloc);
18687	vmaDestroyVirtualBlock(block);
18688	\endcode
18689
18690	\section virtual_allocator_allocation_parameters Allocation parameters
18691
18692	You can attach a custom pointer to each allocation by using vmaSetVirtualAllocationUserData().
18693	Its default value is null.
18694	It can be used to store any data that needs to be associated with that allocation - e.g. an index, a handle, or a pointer to some
18695	larger data structure containing more information. Example:
18696
18697	\code
18698	struct CustomAllocData
18699	{
18700	std::string m_AllocName;
18701	};
18702	CustomAllocData allocData = new CustomAllocData();*
18703	allocData->m_AllocName = "My allocation 1";
18704	vmaSetVirtualAllocationUserData(block, alloc, allocData);
18705	\endcode
18706
18707	The pointer can later be fetched, along with allocation offset and size, by passing the allocation handle to function
18708	vmaGetVirtualAllocationInfo() and inspecting returned structure #VmaVirtualAllocationInfo.
18709	If you allocated a new object to be used as the custom pointer, don't forget to delete that object before freeing the allocation!
18710	Example:
18711
18712	\code
18713	VmaVirtualAllocationInfo allocInfo;
18714	vmaGetVirtualAllocationInfo(block, alloc, &allocInfo);
18715	delete (CustomAllocData)allocInfo.pUserData;*
18716
18717	vmaVirtualFree(block, alloc);
18718	\endcode
18719
18720	\section virtual_allocator_alignment_and_units Alignment and units
18721
18722	It feels natural to express sizes and offsets in bytes.
18723	If an offset of an allocation needs to be aligned to a multiply of some number (e.g. 4 bytes), you can fill optional member
18724	VmaVirtualAllocationCreateInfo::alignment to request it. Example:
18725
18726	\code
18727	VmaVirtualAllocationCreateInfo allocCreateInfo = {};
18728	allocCreateInfo.size = 4096; // 4 KB
18729	allocCreateInfo.alignment = 4; // Returned offset must be a multiply of 4 B
18730
18731	VmaVirtualAllocation alloc;
18732	res = vmaVirtualAllocate(block, &allocCreateInfo, &alloc, nullptr);
18733	\endcode
18734
18735	Alignments of different allocations made from one block may vary.
18736	However, if all alignments and sizes are always multiply of some size e.g. 4 B or `sizeof(MyDataStruct)`,
18737	you can express all sizes, alignments, and offsets in multiples of that size instead of individual bytes.
18738	It might be more convenient, but you need to make sure to use this new unit consistently in all the places:
18739
18740	- VmaVirtualBlockCreateInfo::size
18741	- VmaVirtualAllocationCreateInfo::size and VmaVirtualAllocationCreateInfo::alignment
18742	- Using offset returned by vmaVirtualAllocate() or in VmaVirtualAllocationInfo::offset
18743
18744	\section virtual_allocator_statistics Statistics
18745
18746	You can obtain statistics of a virtual block using vmaGetVirtualBlockStatistics()
18747	(to get brief statistics that are fast to calculate)
18748	or vmaCalculateVirtualBlockStatistics() (to get more detailed statistics, slower to calculate).
18749	The functions fill structures #VmaStatistics, #VmaDetailedStatistics respectively - same as used by the normal Vulkan memory allocator.
18750	Example:
18751
18752	\code
18753	VmaStatistics stats;
18754	vmaGetVirtualBlockStatistics(block, &stats);
18755	printf("My virtual block has %llu bytes used by %u virtual allocations\n",
18756	stats.allocationBytes, stats.allocationCount);
18757	\endcode
18758
18759	You can also request a full list of allocations and free regions as a string in JSON format by calling
18760	vmaBuildVirtualBlockStatsString().
18761	Returned string must be later freed using vmaFreeVirtualBlockStatsString().
18762	The format of this string differs from the one returned by the main Vulkan allocator, but it is similar.
18763
18764	\section virtual_allocator_additional_considerations Additional considerations
18765
18766	The "virtual allocator" functionality is implemented on a level of individual memory blocks.
18767	Keeping track of a whole collection of blocks, allocating new ones when out of free space,
18768	deleting empty ones, and deciding which one to try first for a new allocation must be implemented by the user.
18769
18770	Alternative allocation algorithms are supported, just like in custom pools of the real GPU memory.
18771	See enum #VmaVirtualBlockCreateFlagBits to learn how to specify them (e.g. #VMA_VIRTUAL_BLOCK_CREATE_LINEAR_ALGORITHM_BIT).
18772	You can find their description in chapter \ref custom_memory_pools.
18773	Allocation strategies are also supported.
18774	See enum #VmaVirtualAllocationCreateFlagBits to learn how to specify them (e.g. #VMA_VIRTUAL_ALLOCATION_CREATE_STRATEGY_MIN_TIME_BIT).
18775
18776	Following features are supported only by the allocator of the real GPU memory and not by virtual allocations:
18777	buffer-image granularity, `VMA_DEBUG_MARGIN`, `VMA_MIN_ALIGNMENT`.
18778
18779
18780	\page debugging_memory_usage Debugging incorrect memory usage
18781
18782	If you suspect a bug with memory usage, like usage of uninitialized memory or
18783	memory being overwritten out of bounds of an allocation,
18784	you can use debug features of this library to verify this.
18785
18786	\section debugging_memory_usage_initialization Memory initialization
18787
18788	If you experience a bug with incorrect and nondeterministic data in your program and you suspect uninitialized memory to be used,
18789	you can enable automatic memory initialization to verify this.
18790	To do it, define macro `VMA_DEBUG_INITIALIZE_ALLOCATIONS` to 1.
18791
18792	\code
18793	#define VMA_DEBUG_INITIALIZE_ALLOCATIONS 1
18794	#include "vk_mem_alloc.h"
18795	\endcode
18796
18797	It makes memory of new allocations initialized to bit pattern `0xDCDCDCDC`.
18798	Before an allocation is destroyed, its memory is filled with bit pattern `0xEFEFEFEF`.
18799	Memory is automatically mapped and unmapped if necessary.
18800
18801	If you find these values while debugging your program, good chances are that you incorrectly
18802	read Vulkan memory that is allocated but not initialized, or already freed, respectively.
18803
18804	Memory initialization works only with memory types that are `HOST_VISIBLE` and with allocations that can be mapped.
18805	It works also with dedicated allocations.
18806
18807	\section debugging_memory_usage_margins Margins
18808
18809	By default, allocations are laid out in memory blocks next to each other if possible
18810	(considering required alignment, `bufferImageGranularity`, and `nonCoherentAtomSize`).
18811
18812	![Allocations without margin](../gfx/Margins_1.png)
18813
18814	Define macro `VMA_DEBUG_MARGIN` to some non-zero value (e.g. 16) to enforce specified
18815	number of bytes as a margin after every allocation.
18816
18817	\code
18818	#define VMA_DEBUG_MARGIN 16
18819	#include "vk_mem_alloc.h"
18820	\endcode
18821
18822	![Allocations with margin](../gfx/Margins_2.png)
18823
18824	If your bug goes away after enabling margins, it means it may be caused by memory
18825	being overwritten outside of allocation boundaries. It is not 100% certain though.
18826	Change in application behavior may also be caused by different order and distribution
18827	of allocations across memory blocks after margins are applied.
18828
18829	Margins work with all types of memory.
18830
18831	Margin is applied only to allocations made out of memory blocks and not to dedicated
18832	allocations, which have their own memory block of specific size.
18833	It is thus not applied to allocations made using #VMA_ALLOCATION_CREATE_DEDICATED_MEMORY_BIT flag
18834	or those automatically decided to put into dedicated allocations, e.g. due to its
18835	large size or recommended by VK_KHR_dedicated_allocation extension.
18836
18837	Margins appear in [JSON dump](@ref statistics_json_dump) as part of free space.
18838
18839	Note that enabling margins increases memory usage and fragmentation.
18840
18841	Margins do not apply to \ref virtual_allocator.
18842
18843	\section debugging_memory_usage_corruption_detection Corruption detection
18844
18845	You can additionally define macro `VMA_DEBUG_DETECT_CORRUPTION` to 1 to enable validation
18846	of contents of the margins.
18847
18848	\code
18849	#define VMA_DEBUG_MARGIN 16
18850	#define VMA_DEBUG_DETECT_CORRUPTION 1
18851	#include "vk_mem_alloc.h"
18852	\endcode
18853
18854	When this feature is enabled, number of bytes specified as `VMA_DEBUG_MARGIN`
18855	(it must be multiply of 4) after every allocation is filled with a magic number.
18856	This idea is also know as "canary".
18857	Memory is automatically mapped and unmapped if necessary.
18858
18859	This number is validated automatically when the allocation is destroyed.
18860	If it is not equal to the expected value, `VMA_ASSERT()` is executed.
18861	It clearly means that either CPU or GPU overwritten the memory outside of boundaries of the allocation,
18862	which indicates a serious bug.
18863
18864	You can also explicitly request checking margins of all allocations in all memory blocks
18865	that belong to specified memory types by using function vmaCheckCorruption(),
18866	or in memory blocks that belong to specified custom pool, by using function
18867	vmaCheckPoolCorruption().
18868
18869	Margin validation (corruption detection) works only for memory types that are
18870	`HOST_VISIBLE` and `HOST_COHERENT`.
18871
18872
18873	\page opengl_interop OpenGL Interop
18874
18875	VMA provides some features that help with interoperability with OpenGL.
18876
18877	\section opengl_interop_exporting_memory Exporting memory
18878
18879	If you want to attach `VkExportMemoryAllocateInfoKHR` structure to `pNext` chain of memory allocations made by the library:
18880
18881	It is recommended to create \ref custom_memory_pools for such allocations.
18882	Define and fill in your `VkExportMemoryAllocateInfoKHR` structure and attach it to VmaPoolCreateInfo::pMemoryAllocateNext
18883	while creating the custom pool.
18884	Please note that the structure must remain alive and unchanged for the whole lifetime of the #VmaPool,
18885	not only while creating it, as no copy of the structure is made,
18886	but its original pointer is used for each allocation instead.
18887
18888	If you want to export all memory allocated by the library from certain memory types,
18889	also dedicated allocations or other allocations made from default pools,
18890	an alternative solution is to fill in VmaAllocatorCreateInfo::pTypeExternalMemoryHandleTypes.
18891	It should point to an array with `VkExternalMemoryHandleTypeFlagsKHR` to be automatically passed by the library
18892	through `VkExportMemoryAllocateInfoKHR` on each allocation made from a specific memory type.
18893	Please note that new versions of the library also support dedicated allocations created in custom pools.
18894
18895	You should not mix these two methods in a way that allows to apply both to the same memory type.
18896	Otherwise, `VkExportMemoryAllocateInfoKHR` structure would be attached twice to the `pNext` chain of `VkMemoryAllocateInfo`.
18897
18898
18899	\section opengl_interop_custom_alignment Custom alignment
18900
18901	Buffers or images exported to a different API like OpenGL may require a different alignment,
18902	higher than the one used by the library automatically, queried from functions like `vkGetBufferMemoryRequirements`.
18903	To impose such alignment:
18904
18905	It is recommended to create \ref custom_memory_pools for such allocations.
18906	Set VmaPoolCreateInfo::minAllocationAlignment member to the minimum alignment required for each allocation
18907	to be made out of this pool.
18908	The alignment actually used will be the maximum of this member and the alignment returned for the specific buffer or image
18909	from a function like `vkGetBufferMemoryRequirements`, which is called by VMA automatically.
18910
18911	If you want to create a buffer with a specific minimum alignment out of default pools,
18912	use special function vmaCreateBufferWithAlignment(), which takes additional parameter `minAlignment`.
18913
18914	Note the problem of alignment affects only resources placed inside bigger `VkDeviceMemory` blocks and not dedicated
18915	allocations, as these, by definition, always have alignment = 0 because the resource is bound to the beginning of its dedicated block.
18916	Contrary to Direct3D 12, Vulkan doesn't have a concept of alignment of the entire memory block passed on its allocation.
18917
18918
18919	\page usage_patterns Recommended usage patterns
18920
18921	Vulkan gives great flexibility in memory allocation.
18922	This chapter shows the most common patterns.
18923
18924	See also slides from talk:
18925	[Sawicki, Adam. Advanced Graphics Techniques Tutorial: Memory management in Vulkan and DX12. Game Developers Conference, 2018](https://www.gdcvault.com/play/1025458/Advanced-Graphics-Techniques-Tutorial-New)
18926
18927
18928	\section usage_patterns_gpu_only GPU-only resource
18929
18930	<b>When:</b>
18931	Any resources that you frequently write and read on GPU,
18932	e.g. images used as color attachments (aka "render targets"), depth-stencil attachments,
18933	images/buffers used as storage image/buffer (aka "Unordered Access View (UAV)").
18934
18935	<b>What to do:</b>
18936	Let the library select the optimal memory type, which will likely have `VK_MEMORY_PROPERTY_DEVICE_LOCAL_BIT`.
18937
18938	\code
18939	VkImageCreateInfo imgCreateInfo = { VK_STRUCTURE_TYPE_IMAGE_CREATE_INFO };
18940	imgCreateInfo.imageType = VK_IMAGE_TYPE_2D;
18941	imgCreateInfo.extent.width = 3840;
18942	imgCreateInfo.extent.height = 2160;
18943	imgCreateInfo.extent.depth = 1;
18944	imgCreateInfo.mipLevels = 1;
18945	imgCreateInfo.arrayLayers = 1;
18946	imgCreateInfo.format = VK_FORMAT_R8G8B8A8_UNORM;
18947	imgCreateInfo.tiling = VK_IMAGE_TILING_OPTIMAL;
18948	imgCreateInfo.initialLayout = VK_IMAGE_LAYOUT_UNDEFINED;
18949	imgCreateInfo.usage = VK_IMAGE_USAGE_SAMPLED_BIT \| VK_IMAGE_USAGE_COLOR_ATTACHMENT_BIT;
18950	imgCreateInfo.samples = VK_SAMPLE_COUNT_1_BIT;
18951
18952	VmaAllocationCreateInfo allocCreateInfo = {};
18953	allocCreateInfo.usage = VMA_MEMORY_USAGE_AUTO;
18954	allocCreateInfo.flags = VMA_ALLOCATION_CREATE_DEDICATED_MEMORY_BIT;
18955	allocCreateInfo.priority = 1.0f;
18956
18957	VkImage img;
18958	VmaAllocation alloc;
18959	vmaCreateImage(allocator, &imgCreateInfo, &allocCreateInfo, &img, &alloc, nullptr);
18960	\endcode
18961
18962	<b>Also consider:</b>
18963	Consider creating them as dedicated allocations using #VMA_ALLOCATION_CREATE_DEDICATED_MEMORY_BIT,
18964	especially if they are large or if you plan to destroy and recreate them with different sizes
18965	e.g. when display resolution changes.
18966	Prefer to create such resources first and all other GPU resources (like textures and vertex buffers) later.
18967	When VK_EXT_memory_priority extension is enabled, it is also worth setting high priority to such allocation
18968	to decrease chances to be evicted to system memory by the operating system.
18969
18970	\section usage_patterns_staging_copy_upload Staging copy for upload
18971
18972	<b>When:</b>
18973	A "staging" buffer than you want to map and fill from CPU code, then use as a source od transfer
18974	to some GPU resource.
18975
18976	<b>What to do:</b>
18977	Use flag #VMA_ALLOCATION_CREATE_HOST_ACCESS_SEQUENTIAL_WRITE_BIT.
18978	Let the library select the optimal memory type, which will always have `VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT`.
18979
18980	\code
18981	VkBufferCreateInfo bufCreateInfo = { VK_STRUCTURE_TYPE_BUFFER_CREATE_INFO };
18982	bufCreateInfo.size = 65536;
18983	bufCreateInfo.usage = VK_BUFFER_USAGE_TRANSFER_SRC_BIT;
18984
18985	VmaAllocationCreateInfo allocCreateInfo = {};
18986	allocCreateInfo.usage = VMA_MEMORY_USAGE_AUTO;
18987	allocCreateInfo.flags = VMA_ALLOCATION_CREATE_HOST_ACCESS_SEQUENTIAL_WRITE_BIT \|
18988	VMA_ALLOCATION_CREATE_MAPPED_BIT;
18989
18990	VkBuffer buf;
18991	VmaAllocation alloc;
18992	VmaAllocationInfo allocInfo;
18993	vmaCreateBuffer(allocator, &bufCreateInfo, &allocCreateInfo, &buf, &alloc, &allocInfo);
18994
18995	...
18996
18997	memcpy(allocInfo.pMappedData, myData, myDataSize);
18998	\endcode
18999
19000	<b>Also consider:</b>
19001	You can map the allocation using vmaMapMemory() or you can create it as persistenly mapped
19002	using #VMA_ALLOCATION_CREATE_MAPPED_BIT, as in the example above.
19003
19004
19005	\section usage_patterns_readback Readback
19006
19007	<b>When:</b>
19008	Buffers for data written by or transferred from the GPU that you want to read back on the CPU,
19009	e.g. results of some computations.
19010
19011	<b>What to do:</b>
19012	Use flag #VMA_ALLOCATION_CREATE_HOST_ACCESS_RANDOM_BIT.
19013	Let the library select the optimal memory type, which will always have `VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT`
19014	and `VK_MEMORY_PROPERTY_HOST_CACHED_BIT`.
19015
19016	\code
19017	VkBufferCreateInfo bufCreateInfo = { VK_STRUCTURE_TYPE_BUFFER_CREATE_INFO };
19018	bufCreateInfo.size = 65536;
19019	bufCreateInfo.usage = VK_BUFFER_USAGE_TRANSFER_DST_BIT;
19020
19021	VmaAllocationCreateInfo allocCreateInfo = {};
19022	allocCreateInfo.usage = VMA_MEMORY_USAGE_AUTO;
19023	allocCreateInfo.flags = VMA_ALLOCATION_CREATE_HOST_ACCESS_RANDOM_BIT \|
19024	VMA_ALLOCATION_CREATE_MAPPED_BIT;
19025
19026	VkBuffer buf;
19027	VmaAllocation alloc;
19028	VmaAllocationInfo allocInfo;
19029	vmaCreateBuffer(allocator, &bufCreateInfo, &allocCreateInfo, &buf, &alloc, &allocInfo);
19030
19031	...
19032
19033	const float downloadedData = (const float)allocInfo.pMappedData;
19034	\endcode
19035
19036
19037	\section usage_patterns_advanced_data_uploading Advanced data uploading
19038
19039	For resources that you frequently write on CPU via mapped pointer and
19040	freqnently read on GPU e.g. as a uniform buffer (also called "dynamic"), multiple options are possible:
19041
19042	-# Easiest solution is to have one copy of the resource in `HOST_VISIBLE` memory,
19043	even if it means system RAM (not `DEVICE_LOCAL`) on systems with a discrete graphics card,
19044	and make the device reach out to that resource directly.
19045	- Reads performed by the device will then go through PCI Express bus.
19046	The performace of this access may be limited, but it may be fine depending on the size
19047	of this resource (whether it is small enough to quickly end up in GPU cache) and the sparsity
19048	of access.
19049	-# On systems with unified memory (e.g. AMD APU or Intel integrated graphics, mobile chips),
19050	a memory type may be available that is both `HOST_VISIBLE` (available for mapping) and `DEVICE_LOCAL`
19051	(fast to access from the GPU). Then, it is likely the best choice for such type of resource.
19052	-# Systems with a discrete graphics card and separate video memory may or may not expose
19053	a memory type that is both `HOST_VISIBLE` and `DEVICE_LOCAL`, also known as Base Address Register (BAR).
19054	If they do, it represents a piece of VRAM (or entire VRAM, if ReBAR is enabled in the motherboard BIOS)
19055	that is available to CPU for mapping.
19056	- Writes performed by the host to that memory go through PCI Express bus.
19057	The performance of these writes may be limited, but it may be fine, especially on PCIe 4.0,
19058	as long as rules of using uncached and write-combined memory are followed - only sequential writes and no reads.
19059	-# Finally, you may need or prefer to create a separate copy of the resource in `DEVICE_LOCAL` memory,
19060	a separate "staging" copy in `HOST_VISIBLE` memory and perform an explicit transfer command between them.
19061
19062	Thankfully, VMA offers an aid to create and use such resources in the the way optimal
19063	for the current Vulkan device. To help the library make the best choice,
19064	use flag #VMA_ALLOCATION_CREATE_HOST_ACCESS_SEQUENTIAL_WRITE_BIT together with
19065	#VMA_ALLOCATION_CREATE_HOST_ACCESS_ALLOW_TRANSFER_INSTEAD_BIT.
19066	It will then prefer a memory type that is both `DEVICE_LOCAL` and `HOST_VISIBLE` (integrated memory or BAR),
19067	but if no such memory type is available or allocation from it fails
19068	(PC graphics cards have only 256 MB of BAR by default, unless ReBAR is supported and enabled in BIOS),
19069	it will fall back to `DEVICE_LOCAL` memory for fast GPU access.
19070	It is then up to you to detect that the allocation ended up in a memory type that is not `HOST_VISIBLE`,
19071	so you need to create another "staging" allocation and perform explicit transfers.
19072
19073	\code
19074	VkBufferCreateInfo bufCreateInfo = { VK_STRUCTURE_TYPE_BUFFER_CREATE_INFO };
19075	bufCreateInfo.size = 65536;
19076	bufCreateInfo.usage = VK_BUFFER_USAGE_UNIFORM_BUFFER_BIT \| VK_BUFFER_USAGE_TRANSFER_DST_BIT;
19077
19078	VmaAllocationCreateInfo allocCreateInfo = {};
19079	allocCreateInfo.usage = VMA_MEMORY_USAGE_AUTO;
19080	allocCreateInfo.flags = VMA_ALLOCATION_CREATE_HOST_ACCESS_SEQUENTIAL_WRITE_BIT \|
19081	VMA_ALLOCATION_CREATE_HOST_ACCESS_ALLOW_TRANSFER_INSTEAD_BIT \|
19082	VMA_ALLOCATION_CREATE_MAPPED_BIT;
19083
19084	VkBuffer buf;
19085	VmaAllocation alloc;
19086	VmaAllocationInfo allocInfo;
19087	vmaCreateBuffer(allocator, &bufCreateInfo, &allocCreateInfo, &buf, &alloc, &allocInfo);
19088
19089	VkMemoryPropertyFlags memPropFlags;
19090	vmaGetAllocationMemoryProperties(allocator, alloc, &memPropFlags);
19091
19092	if(memPropFlags & VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT)
19093	{
19094	// Allocation ended up in a mappable memory and is already mapped - write to it directly.
19095
19096	// [Executed in runtime]:
19097	memcpy(allocInfo.pMappedData, myData, myDataSize);
19098	}
19099	else
19100	{
19101	// Allocation ended up in a non-mappable memory - need to transfer.
19102	VkBufferCreateInfo stagingBufCreateInfo = { VK_STRUCTURE_TYPE_BUFFER_CREATE_INFO };
19103	stagingBufCreateInfo.size = 65536;
19104	stagingBufCreateInfo.usage = VK_BUFFER_USAGE_TRANSFER_SRC_BIT;
19105
19106	VmaAllocationCreateInfo stagingAllocCreateInfo = {};
19107	stagingAllocCreateInfo.usage = VMA_MEMORY_USAGE_AUTO;
19108	stagingAllocCreateInfo.flags = VMA_ALLOCATION_CREATE_HOST_ACCESS_SEQUENTIAL_WRITE_BIT \|
19109	VMA_ALLOCATION_CREATE_MAPPED_BIT;
19110
19111	VkBuffer stagingBuf;
19112	VmaAllocation stagingAlloc;
19113	VmaAllocationInfo stagingAllocInfo;
19114	vmaCreateBuffer(allocator, &stagingBufCreateInfo, &stagingAllocCreateInfo,
19115	&stagingBuf, &stagingAlloc, stagingAllocInfo);
19116
19117	// [Executed in runtime]:
19118	memcpy(stagingAllocInfo.pMappedData, myData, myDataSize);
19119	//vkCmdPipelineBarrier: VK_ACCESS_HOST_WRITE_BIT --> VK_ACCESS_TRANSFER_READ_BIT
19120	VkBufferCopy bufCopy = {
19121	0, // srcOffset
19122	0, // dstOffset,
19123	myDataSize); // size
19124	vkCmdCopyBuffer(cmdBuf, stagingBuf, buf, 1, &bufCopy);
19125	}
19126	\endcode
19127
19128	\section usage_patterns_other_use_cases Other use cases
19129
19130	Here are some other, less obvious use cases and their recommended settings:
19131
19132	- An image that is used only as transfer source and destination, but it should stay on the device,
19133	as it is used to temporarily store a copy of some texture, e.g. from the current to the next frame,
19134	for temporal antialiasing or other temporal effects.
19135	- Use `VkImageCreateInfo::usage = VK_IMAGE_USAGE_TRANSFER_SRC_BIT \| VK_IMAGE_USAGE_TRANSFER_DST_BIT`
19136	- Use VmaAllocationCreateInfo::usage = #VMA_MEMORY_USAGE_AUTO
19137	- An image that is used only as transfer source and destination, but it should be placed
19138	in the system RAM despite it doesn't need to be mapped, because it serves as a "swap" copy to evict
19139	least recently used textures from VRAM.
19140	- Use `VkImageCreateInfo::usage = VK_IMAGE_USAGE_TRANSFER_SRC_BIT \| VK_IMAGE_USAGE_TRANSFER_DST_BIT`
19141	- Use VmaAllocationCreateInfo::usage = #VMA_MEMORY_USAGE_AUTO_PREFER_HOST,
19142	as VMA needs a hint here to differentiate from the previous case.
19143	- A buffer that you want to map and write from the CPU, directly read from the GPU
19144	(e.g. as a uniform or vertex buffer), but you have a clear preference to place it in device or
19145	host memory due to its large size.
19146	- Use `VkBufferCreateInfo::usage = VK_BUFFER_USAGE_UNIFORM_BUFFER_BIT`
19147	- Use VmaAllocationCreateInfo::usage = #VMA_MEMORY_USAGE_AUTO_PREFER_DEVICE or #VMA_MEMORY_USAGE_AUTO_PREFER_HOST
19148	- Use VmaAllocationCreateInfo::flags = #VMA_ALLOCATION_CREATE_HOST_ACCESS_SEQUENTIAL_WRITE_BIT
19149
19150
19151	\page configuration Configuration
19152
19153	Please check "CONFIGURATION SECTION" in the code to find macros that you can define
19154	before each include of this file or change directly in this file to provide
19155	your own implementation of basic facilities like assert, `min()` and `max()` functions,
19156	mutex, atomic etc.
19157	The library uses its own implementation of containers by default, but you can switch to using
19158	STL containers instead.
19159
19160	For example, define `VMA_ASSERT(expr)` before including the library to provide
19161	custom implementation of the assertion, compatible with your project.
19162	By default it is defined to standard C `assert(expr)` in `_DEBUG` configuration
19163	and empty otherwise.
19164
19165	\section config_Vulkan_functions Pointers to Vulkan functions
19166
19167	There are multiple ways to import pointers to Vulkan functions in the library.
19168	In the simplest case you don't need to do anything.
19169	If the compilation or linking of your program or the initialization of the #VmaAllocator
19170	doesn't work for you, you can try to reconfigure it.
19171
19172	First, the allocator tries to fetch pointers to Vulkan functions linked statically,
19173	like this:
19174
19175	\code
19176	m_VulkanFunctions.vkAllocateMemory = (PFN_vkAllocateMemory)vkAllocateMemory;
19177	\endcode
19178
19179	If you want to disable this feature, set configuration macro: `#define VMA_STATIC_VULKAN_FUNCTIONS 0`.
19180
19181	Second, you can provide the pointers yourself by setting member VmaAllocatorCreateInfo::pVulkanFunctions.
19182	You can fetch them e.g. using functions `vkGetInstanceProcAddr` and `vkGetDeviceProcAddr` or
19183	by using a helper library like [volk](https://github.com/zeux/volk).
19184
19185	Third, VMA tries to fetch remaining pointers that are still null by calling
19186	`vkGetInstanceProcAddr` and `vkGetDeviceProcAddr` on its own.
19187	You need to only fill in VmaVulkanFunctions::vkGetInstanceProcAddr and VmaVulkanFunctions::vkGetDeviceProcAddr.
19188	Other pointers will be fetched automatically.
19189	If you want to disable this feature, set configuration macro: `#define VMA_DYNAMIC_VULKAN_FUNCTIONS 0`.
19190
19191	Finally, all the function pointers required by the library (considering selected
19192	Vulkan version and enabled extensions) are checked with `VMA_ASSERT` if they are not null.
19193
19194
19195	\section custom_memory_allocator Custom host memory allocator
19196
19197	If you use custom allocator for CPU memory rather than default operator `new`
19198	and `delete` from C++, you can make this library using your allocator as well
19199	by filling optional member VmaAllocatorCreateInfo::pAllocationCallbacks. These
19200	functions will be passed to Vulkan, as well as used by the library itself to
19201	make any CPU-side allocations.
19202
19203	\section allocation_callbacks Device memory allocation callbacks
19204
19205	The library makes calls to `vkAllocateMemory()` and `vkFreeMemory()` internally.
19206	You can setup callbacks to be informed about these calls, e.g. for the purpose
19207	of gathering some statistics. To do it, fill optional member
19208	VmaAllocatorCreateInfo::pDeviceMemoryCallbacks.
19209
19210	\section heap_memory_limit Device heap memory limit
19211
19212	When device memory of certain heap runs out of free space, new allocations may
19213	fail (returning error code) or they may succeed, silently pushing some existing_
19214	memory blocks from GPU VRAM to system RAM (which degrades performance). This
19215	behavior is implementation-dependent - it depends on GPU vendor and graphics
19216	driver.
19217
19218	On AMD cards it can be controlled while creating Vulkan device object by using
19219	VK_AMD_memory_overallocation_behavior extension, if available.
19220
19221	Alternatively, if you want to test how your program behaves with limited amount of Vulkan device
19222	memory available without switching your graphics card to one that really has
19223	smaller VRAM, you can use a feature of this library intended for this purpose.
19224	To do it, fill optional member VmaAllocatorCreateInfo::pHeapSizeLimit.
19225
19226
19227
19228	\page vk_khr_dedicated_allocation VK_KHR_dedicated_allocation
19229
19230	VK_KHR_dedicated_allocation is a Vulkan extension which can be used to improve
19231	performance on some GPUs. It augments Vulkan API with possibility to query
19232	driver whether it prefers particular buffer or image to have its own, dedicated
19233	allocation (separate `VkDeviceMemory` block) for better efficiency - to be able
19234	to do some internal optimizations. The extension is supported by this library.
19235	It will be used automatically when enabled.
19236
19237	It has been promoted to core Vulkan 1.1, so if you use eligible Vulkan version
19238	and inform VMA about it by setting VmaAllocatorCreateInfo::vulkanApiVersion,
19239	you are all set.
19240
19241	Otherwise, if you want to use it as an extension:
19242
19243	1 . When creating Vulkan device, check if following 2 device extensions are
19244	supported (call `vkEnumerateDeviceExtensionProperties()`).
19245	If yes, enable them (fill `VkDeviceCreateInfo::ppEnabledExtensionNames`).
19246
19247	- VK_KHR_get_memory_requirements2
19248	- VK_KHR_dedicated_allocation
19249
19250	If you enabled these extensions:
19251
19252	2 . Use #VMA_ALLOCATOR_CREATE_KHR_DEDICATED_ALLOCATION_BIT flag when creating
19253	your #VmaAllocator to inform the library that you enabled required extensions
19254	and you want the library to use them.
19255
19256	\code
19257	allocatorInfo.flags \|= VMA_ALLOCATOR_CREATE_KHR_DEDICATED_ALLOCATION_BIT;
19258
19259	vmaCreateAllocator(&allocatorInfo, &allocator);
19260	\endcode
19261
19262	That is all. The extension will be automatically used whenever you create a
19263	buffer using vmaCreateBuffer() or image using vmaCreateImage().
19264
19265	When using the extension together with Vulkan Validation Layer, you will receive
19266	warnings like this:
19267
19268	_vkBindBufferMemory(): Binding memory to buffer 0x33 but vkGetBufferMemoryRequirements() has not been called on that buffer._
19269
19270	It is OK, you should just ignore it. It happens because you use function
19271	`vkGetBufferMemoryRequirements2KHR()` instead of standard
19272	`vkGetBufferMemoryRequirements()`, while the validation layer seems to be
19273	unaware of it.
19274
19275	To learn more about this extension, see:
19276
19277	- [VK_KHR_dedicated_allocation in Vulkan specification](https://www.khronos.org/registry/vulkan/specs/1.2-extensions/html/chap50.html#VK_KHR_dedicated_allocation)
19278	- [VK_KHR_dedicated_allocation unofficial manual](http://asawicki.info/articles/VK_KHR_dedicated_allocation.php5)
19279
19280
19281
19282	\page vk_ext_memory_priority VK_EXT_memory_priority
19283
19284	VK_EXT_memory_priority is a device extension that allows to pass additional "priority"
19285	value to Vulkan memory allocations that the implementation may use prefer certain
19286	buffers and images that are critical for performance to stay in device-local memory
19287	in cases when the memory is over-subscribed, while some others may be moved to the system memory.
19288
19289	VMA offers convenient usage of this extension.
19290	If you enable it, you can pass "priority" parameter when creating allocations or custom pools
19291	and the library automatically passes the value to Vulkan using this extension.
19292
19293	If you want to use this extension in connection with VMA, follow these steps:
19294
19295	\section vk_ext_memory_priority_initialization Initialization
19296
19297	1) Call `vkEnumerateDeviceExtensionProperties` for the physical device.
19298	Check if the extension is supported - if returned array of `VkExtensionProperties` contains "VK_EXT_memory_priority".
19299
19300	2) Call `vkGetPhysicalDeviceFeatures2` for the physical device instead of old `vkGetPhysicalDeviceFeatures`.
19301	Attach additional structure `VkPhysicalDeviceMemoryPriorityFeaturesEXT` to `VkPhysicalDeviceFeatures2::pNext` to be returned.
19302	Check if the device feature is really supported - check if `VkPhysicalDeviceMemoryPriorityFeaturesEXT::memoryPriority` is true.
19303
19304	3) While creating device with `vkCreateDevice`, enable this extension - add "VK_EXT_memory_priority"
19305	to the list passed as `VkDeviceCreateInfo::ppEnabledExtensionNames`.
19306
19307	4) While creating the device, also don't set `VkDeviceCreateInfo::pEnabledFeatures`.
19308	Fill in `VkPhysicalDeviceFeatures2` structure instead and pass it as `VkDeviceCreateInfo::pNext`.
19309	Enable this device feature - attach additional structure `VkPhysicalDeviceMemoryPriorityFeaturesEXT` to
19310	`VkPhysicalDeviceFeatures2::pNext` chain and set its member `memoryPriority` to `VK_TRUE`.
19311
19312	5) While creating #VmaAllocator with vmaCreateAllocator() inform VMA that you
19313	have enabled this extension and feature - add #VMA_ALLOCATOR_CREATE_EXT_MEMORY_PRIORITY_BIT
19314	to VmaAllocatorCreateInfo::flags.
19315
19316	\section vk_ext_memory_priority_usage Usage
19317
19318	When using this extension, you should initialize following member:
19319
19320	- VmaAllocationCreateInfo::priority when creating a dedicated allocation with #VMA_ALLOCATION_CREATE_DEDICATED_MEMORY_BIT.
19321	- VmaPoolCreateInfo::priority when creating a custom pool.
19322
19323	It should be a floating-point value between `0.0f` and `1.0f`, where recommended default is `0.5f`.
19324	Memory allocated with higher value can be treated by the Vulkan implementation as higher priority
19325	and so it can have lower chances of being pushed out to system memory, experiencing degraded performance.
19326
19327	It might be a good idea to create performance-critical resources like color-attachment or depth-stencil images
19328	as dedicated and set high priority to them. For example:
19329
19330	\code
19331	VkImageCreateInfo imgCreateInfo = { VK_STRUCTURE_TYPE_IMAGE_CREATE_INFO };
19332	imgCreateInfo.imageType = VK_IMAGE_TYPE_2D;
19333	imgCreateInfo.extent.width = 3840;
19334	imgCreateInfo.extent.height = 2160;
19335	imgCreateInfo.extent.depth = 1;
19336	imgCreateInfo.mipLevels = 1;
19337	imgCreateInfo.arrayLayers = 1;
19338	imgCreateInfo.format = VK_FORMAT_R8G8B8A8_UNORM;
19339	imgCreateInfo.tiling = VK_IMAGE_TILING_OPTIMAL;
19340	imgCreateInfo.initialLayout = VK_IMAGE_LAYOUT_UNDEFINED;
19341	imgCreateInfo.usage = VK_IMAGE_USAGE_SAMPLED_BIT \| VK_IMAGE_USAGE_COLOR_ATTACHMENT_BIT;
19342	imgCreateInfo.samples = VK_SAMPLE_COUNT_1_BIT;
19343
19344	VmaAllocationCreateInfo allocCreateInfo = {};
19345	allocCreateInfo.usage = VMA_MEMORY_USAGE_AUTO;
19346	allocCreateInfo.flags = VMA_ALLOCATION_CREATE_DEDICATED_MEMORY_BIT;
19347	allocCreateInfo.priority = 1.0f;
19348
19349	VkImage img;
19350	VmaAllocation alloc;
19351	vmaCreateImage(allocator, &imgCreateInfo, &allocCreateInfo, &img, &alloc, nullptr);
19352	\endcode
19353
19354	`priority` member is ignored in the following situations:
19355
19356	- Allocations created in custom pools: They inherit the priority, along with all other allocation parameters
19357	from the parametrs passed in #VmaPoolCreateInfo when the pool was created.
19358	- Allocations created in default pools: They inherit the priority from the parameters
19359	VMA used when creating default pools, which means `priority == 0.5f`.
19360
19361
19362	\page vk_amd_device_coherent_memory VK_AMD_device_coherent_memory
19363
19364	VK_AMD_device_coherent_memory is a device extension that enables access to
19365	additional memory types with `VK_MEMORY_PROPERTY_DEVICE_COHERENT_BIT_AMD` and
19366	`VK_MEMORY_PROPERTY_DEVICE_UNCACHED_BIT_AMD` flag. It is useful mostly for
19367	allocation of buffers intended for writing "breadcrumb markers" in between passes
19368	or draw calls, which in turn are useful for debugging GPU crash/hang/TDR cases.
19369
19370	When the extension is available but has not been enabled, Vulkan physical device
19371	still exposes those memory types, but their usage is forbidden. VMA automatically
19372	takes care of that - it returns `VK_ERROR_FEATURE_NOT_PRESENT` when an attempt
19373	to allocate memory of such type is made.
19374
19375	If you want to use this extension in connection with VMA, follow these steps:
19376
19377	\section vk_amd_device_coherent_memory_initialization Initialization
19378
19379	1) Call `vkEnumerateDeviceExtensionProperties` for the physical device.
19380	Check if the extension is supported - if returned array of `VkExtensionProperties` contains "VK_AMD_device_coherent_memory".
19381
19382	2) Call `vkGetPhysicalDeviceFeatures2` for the physical device instead of old `vkGetPhysicalDeviceFeatures`.
19383	Attach additional structure `VkPhysicalDeviceCoherentMemoryFeaturesAMD` to `VkPhysicalDeviceFeatures2::pNext` to be returned.
19384	Check if the device feature is really supported - check if `VkPhysicalDeviceCoherentMemoryFeaturesAMD::deviceCoherentMemory` is true.
19385
19386	3) While creating device with `vkCreateDevice`, enable this extension - add "VK_AMD_device_coherent_memory"
19387	to the list passed as `VkDeviceCreateInfo::ppEnabledExtensionNames`.
19388
19389	4) While creating the device, also don't set `VkDeviceCreateInfo::pEnabledFeatures`.
19390	Fill in `VkPhysicalDeviceFeatures2` structure instead and pass it as `VkDeviceCreateInfo::pNext`.
19391	Enable this device feature - attach additional structure `VkPhysicalDeviceCoherentMemoryFeaturesAMD` to
19392	`VkPhysicalDeviceFeatures2::pNext` and set its member `deviceCoherentMemory` to `VK_TRUE`.
19393
19394	5) While creating #VmaAllocator with vmaCreateAllocator() inform VMA that you
19395	have enabled this extension and feature - add #VMA_ALLOCATOR_CREATE_AMD_DEVICE_COHERENT_MEMORY_BIT
19396	to VmaAllocatorCreateInfo::flags.
19397
19398	\section vk_amd_device_coherent_memory_usage Usage
19399
19400	After following steps described above, you can create VMA allocations and custom pools
19401	out of the special `DEVICE_COHERENT` and `DEVICE_UNCACHED` memory types on eligible
19402	devices. There are multiple ways to do it, for example:
19403
19404	- You can request or prefer to allocate out of such memory types by adding
19405	`VK_MEMORY_PROPERTY_DEVICE_UNCACHED_BIT_AMD` to VmaAllocationCreateInfo::requiredFlags
19406	or VmaAllocationCreateInfo::preferredFlags. Those flags can be freely mixed with
19407	other ways of \ref choosing_memory_type, like setting VmaAllocationCreateInfo::usage.
19408	- If you manually found memory type index to use for this purpose, force allocation
19409	from this specific index by setting VmaAllocationCreateInfo::memoryTypeBits `= 1u << index`.
19410
19411	\section vk_amd_device_coherent_memory_more_information More information
19412
19413	To learn more about this extension, see [VK_AMD_device_coherent_memory in Vulkan specification](https://www.khronos.org/registry/vulkan/specs/1.2-extensions/man/html/VK_AMD_device_coherent_memory.html)
19414
19415	Example use of this extension can be found in the code of the sample and test suite
19416	accompanying this library.
19417
19418
19419	\page enabling_buffer_device_address Enabling buffer device address
19420
19421	Device extension VK_KHR_buffer_device_address
19422	allow to fetch raw GPU pointer to a buffer and pass it for usage in a shader code.
19423	It has been promoted to core Vulkan 1.2.
19424
19425	If you want to use this feature in connection with VMA, follow these steps:
19426
19427	\section enabling_buffer_device_address_initialization Initialization
19428
19429	1) (For Vulkan version < 1.2) Call `vkEnumerateDeviceExtensionProperties` for the physical device.
19430	Check if the extension is supported - if returned array of `VkExtensionProperties` contains
19431	"VK_KHR_buffer_device_address".
19432
19433	2) Call `vkGetPhysicalDeviceFeatures2` for the physical device instead of old `vkGetPhysicalDeviceFeatures`.
19434	Attach additional structure `VkPhysicalDeviceBufferDeviceAddressFeatures` to `VkPhysicalDeviceFeatures2::pNext` to be returned.*
19435	Check if the device feature is really supported - check if `VkPhysicalDeviceBufferDeviceAddressFeatures::bufferDeviceAddress` is true.
19436
19437	3) (For Vulkan version < 1.2) While creating device with `vkCreateDevice`, enable this extension - add
19438	"VK_KHR_buffer_device_address" to the list passed as `VkDeviceCreateInfo::ppEnabledExtensionNames`.
19439
19440	4) While creating the device, also don't set `VkDeviceCreateInfo::pEnabledFeatures`.
19441	Fill in `VkPhysicalDeviceFeatures2` structure instead and pass it as `VkDeviceCreateInfo::pNext`.
19442	Enable this device feature - attach additional structure `VkPhysicalDeviceBufferDeviceAddressFeatures` to*
19443	`VkPhysicalDeviceFeatures2::pNext` and set its member `bufferDeviceAddress` to `VK_TRUE`.
19444
19445	5) While creating #VmaAllocator with vmaCreateAllocator() inform VMA that you
19446	have enabled this feature - add #VMA_ALLOCATOR_CREATE_BUFFER_DEVICE_ADDRESS_BIT
19447	to VmaAllocatorCreateInfo::flags.
19448
19449	\section enabling_buffer_device_address_usage Usage
19450
19451	After following steps described above, you can create buffers with `VK_BUFFER_USAGE_SHADER_DEVICE_ADDRESS_BIT` using VMA.*
19452	The library automatically adds `VK_MEMORY_ALLOCATE_DEVICE_ADDRESS_BIT` to*
19453	allocated memory blocks wherever it might be needed.
19454
19455	Please note that the library supports only `VK_BUFFER_USAGE_SHADER_DEVICE_ADDRESS_BIT`.*
19456	The second part of this functionality related to "capture and replay" is not supported,
19457	as it is intended for usage in debugging tools like RenderDoc, not in everyday Vulkan usage.
19458
19459	\section enabling_buffer_device_address_more_information More information
19460
19461	To learn more about this extension, see [VK_KHR_buffer_device_address in Vulkan specification](https://www.khronos.org/registry/vulkan/specs/1.2-extensions/html/chap46.html#VK_KHR_buffer_device_address)
19462
19463	Example use of this extension can be found in the code of the sample and test suite
19464	accompanying this library.
19465
19466	\page general_considerations General considerations
19467
19468	\section general_considerations_thread_safety Thread safety
19469
19470	- The library has no global state, so separate #VmaAllocator objects can be used
19471	independently.
19472	There should be no need to create multiple such objects though - one per `VkDevice` is enough.
19473	- By default, all calls to functions that take #VmaAllocator as first parameter
19474	are safe to call from multiple threads simultaneously because they are
19475	synchronized internally when needed.
19476	This includes allocation and deallocation from default memory pool, as well as custom #VmaPool.
19477	- When the allocator is created with #VMA_ALLOCATOR_CREATE_EXTERNALLY_SYNCHRONIZED_BIT
19478	flag, calls to functions that take such #VmaAllocator object must be
19479	synchronized externally.
19480	- Access to a #VmaAllocation object must be externally synchronized. For example,
19481	you must not call vmaGetAllocationInfo() and vmaMapMemory() from different
19482	threads at the same time if you pass the same #VmaAllocation object to these
19483	functions.
19484	- #VmaVirtualBlock is not safe to be used from multiple threads simultaneously.
19485
19486	\section general_considerations_versioning_and_compatibility Versioning and compatibility
19487
19488	The library uses [Semantic Versioning](https://semver.org/),
19489	which means version numbers follow convention: Major.Minor.Patch (e.g. 2.3.0), where:
19490
19491	- Incremented Patch version means a release is backward- and forward-compatible,
19492	introducing only some internal improvements, bug fixes, optimizations etc.
19493	or changes that are out of scope of the official API described in this documentation.
19494	- Incremented Minor version means a release is backward-compatible,
19495	so existing code that uses the library should continue to work, while some new
19496	symbols could have been added: new structures, functions, new values in existing
19497	enums and bit flags, new structure members, but not new function parameters.
19498	- Incrementing Major version means a release could break some backward compatibility.
19499
19500	All changes between official releases are documented in file "CHANGELOG.md".
19501
19502	\warning Backward compatiblity is considered on the level of C++ source code, not binary linkage.
19503	Adding new members to existing structures is treated as backward compatible if initializing
19504	the new members to binary zero results in the old behavior.
19505	You should always fully initialize all library structures to zeros and not rely on their
19506	exact binary size.
19507
19508	\section general_considerations_validation_layer_warnings Validation layer warnings
19509
19510	When using this library, you can meet following types of warnings issued by
19511	Vulkan validation layer. They don't necessarily indicate a bug, so you may need
19512	to just ignore them.
19513
19514	- vkBindBufferMemory(): Binding memory to buffer 0xeb8e4 but vkGetBufferMemoryRequirements() has not been called on that buffer.
19515	- It happens when VK_KHR_dedicated_allocation extension is enabled.
19516	`vkGetBufferMemoryRequirements2KHR` function is used instead, while validation layer seems to be unaware of it.
19517	- Mapping an image with layout VK_IMAGE_LAYOUT_DEPTH_STENCIL_ATTACHMENT_OPTIMAL can result in undefined behavior if this memory is used by the device. Only GENERAL or PREINITIALIZED should be used.
19518	- It happens when you map a buffer or image, because the library maps entire
19519	`VkDeviceMemory` block, where different types of images and buffers may end
19520	up together, especially on GPUs with unified memory like Intel.
19521	- Non-linear image 0xebc91 is aliased with linear buffer 0xeb8e4 which may indicate a bug.
19522	- It may happen when you use [defragmentation](@ref defragmentation).
19523
19524	\section general_considerations_allocation_algorithm Allocation algorithm
19525
19526	The library uses following algorithm for allocation, in order:
19527
19528	-# Try to find free range of memory in existing blocks.
19529	-# If failed, try to create a new block of `VkDeviceMemory`, with preferred block size.
19530	-# If failed, try to create such block with size / 2, size / 4, size / 8.
19531	-# If failed, try to allocate separate `VkDeviceMemory` for this allocation,
19532	just like when you use #VMA_ALLOCATION_CREATE_DEDICATED_MEMORY_BIT.
19533	-# If failed, choose other memory type that meets the requirements specified in
19534	VmaAllocationCreateInfo and go to point 1.
19535	-# If failed, return `VK_ERROR_OUT_OF_DEVICE_MEMORY`.
19536
19537	\section general_considerations_features_not_supported Features not supported
19538
19539	Features deliberately excluded from the scope of this library:
19540
19541	-# Data transfer.* Uploading (streaming) and downloading data of buffers and images*
19542	between CPU and GPU memory and related synchronization is responsibility of the user.
19543	Defining some "texture" object that would automatically stream its data from a
19544	staging copy in CPU memory to GPU memory would rather be a feature of another,
19545	higher-level library implemented on top of VMA.
19546	VMA doesn't record any commands to a `VkCommandBuffer`. It just allocates memory.
19547	-# Recreation of buffers and images.* Although the library has functions for*
19548	buffer and image creation: vmaCreateBuffer(), vmaCreateImage(), you need to
19549	recreate these objects yourself after defragmentation. That is because the big
19550	structures `VkBufferCreateInfo`, `VkImageCreateInfo` are not stored in
19551	#VmaAllocation object.
19552	-# Handling CPU memory allocation failures.* When dynamically creating small C++*
19553	objects in CPU memory (not Vulkan memory), allocation failures are not checked
19554	and handled gracefully, because that would complicate code significantly and
19555	is usually not needed in desktop PC applications anyway.
19556	Success of an allocation is just checked with an assert.
19557	-# Code free of any compiler warnings.* Maintaining the library to compile and*
19558	work correctly on so many different platforms is hard enough. Being free of
19559	any warnings, on any version of any compiler, is simply not feasible.
19560	There are many preprocessor macros that make some variables unused, function parameters unreferenced,
19561	or conditional expressions constant in some configurations.
19562	The code of this library should not be bigger or more complicated just to silence these warnings.
19563	It is recommended to disable such warnings instead.
19564	-# This is a C++ library with C interface. Bindings or ports to any other programming languages* are welcome as external projects but*
19565	are not going to be included into this repository.
19566	*/
19567

Browse the source code of Godot/thirdparty/vulkan/vk_mem_alloc.h