1/****************************************************************************
2**
3** Copyright (C) 2019 The Qt Company Ltd.
4** Contact: http://www.qt.io/licensing/
5**
6** This file is part of the Qt Gui module
7**
8** $QT_BEGIN_LICENSE:LGPL3$
9** Commercial License Usage
10** Licensees holding valid commercial Qt licenses may use this file in
11** accordance with the commercial license agreement provided with the
12** Software or, alternatively, in accordance with the terms contained in
13** a written agreement between you and The Qt Company. For licensing terms
14** and conditions see http://www.qt.io/terms-conditions. For further
15** information use the contact form at http://www.qt.io/contact-us.
16**
17** GNU Lesser General Public License Usage
18** Alternatively, this file may be used under the terms of the GNU Lesser
19** General Public License version 3 as published by the Free Software
20** Foundation and appearing in the file LICENSE.LGPLv3 included in the
21** packaging of this file. Please review the following information to
22** ensure the GNU Lesser General Public License version 3 requirements
23** will be met: https://www.gnu.org/licenses/lgpl.html.
24**
25** GNU General Public License Usage
26** Alternatively, this file may be used under the terms of the GNU
27** General Public License version 2.0 or later as published by the Free
28** Software Foundation and appearing in the file LICENSE.GPL included in
29** the packaging of this file. Please review the following information to
30** ensure the GNU General Public License version 2.0 requirements will be
31** met: http://www.gnu.org/licenses/gpl-2.0.html.
32**
33** $QT_END_LICENSE$
34**
35****************************************************************************/
36
37#include "qshader_p_p.h"
38#include <QDataStream>
39#include <QBuffer>
40
41QT_BEGIN_NAMESPACE
42
43/*!
44 \class QShader
45 \internal
46 \inmodule QtGui
47
48 \brief Contains multiple versions of a shader translated to multiple shading languages,
49 together with reflection metadata.
50
51 QShader is the entry point to shader code in the graphics API agnostic
52 Qt world. Instead of using GLSL shader sources, as was the custom with Qt
53 5.x, new graphics systems with backends for multiple graphics APIs, such
54 as, Vulkan, Metal, Direct3D, and OpenGL, take QShader as their input
55 whenever a shader needs to be specified.
56
57 A QShader instance is empty and thus invalid by default. To get a useful
58 instance, the two typical methods are:
59
60 \list
61
62 \li Generate the contents offline, during build time or earlier, using the
63 \c qsb command line tool. The result is a binary file that is shipped with
64 the application, read via QIODevice::readAll(), and then deserialized via
65 fromSerialized(). For more information, see QShaderBaker.
66
67 \li Generate at run time via QShaderBaker. This is an expensive operation,
68 but allows applications to use user-provided or dynamically generated
69 shader source strings.
70
71 \endlist
72
73 When used together with the Qt Rendering Hardware Interface and its
74 classes, like QRhiGraphicsPipeline, no further action is needed from the
75 application's side as these classes are prepared to consume a QShader
76 whenever a shader needs to be specified for a given stage of the graphics
77 pipeline.
78
79 Alternatively, applications can access
80
81 \list
82
83 \li the source or byte code for any of the shading language versions that
84 are included in the QShader,
85
86 \li the name of the entry point for the shader,
87
88 \li the reflection metadata containing a description of the shader's
89 inputs, outputs and resources like uniform blocks. This is essential when
90 an application or framework needs to discover the inputs of a shader at
91 runtime due to not having advance knowledge of the vertex attributes or the
92 layout of the uniform buffers used by the shader.
93
94 \endlist
95
96 QShader makes no assumption about the shading language that was used
97 as the source for generating the various versions and variants that are
98 included in it.
99
100 QShader uses implicit sharing similarly to many core Qt types, and so
101 can be returned or passed by value. Detach happens implicitly when calling
102 a setter.
103
104 For reference, QRhi expects that a QShader suitable for all its
105 backends contains at least the following:
106
107 \list
108
109 \li SPIR-V 1.0 bytecode suitable for Vulkan 1.0 or newer
110
111 \li GLSL/ES 100 source code suitable for OpenGL ES 2.0 or newer
112
113 \li GLSL 120 source code suitable for OpenGL 2.1
114
115 \li HLSL Shader Model 5.0 source code or the corresponding DXBC bytecode suitable for Direct3D 11
116
117 \li Metal Shading Language 1.2 source code or the corresponding bytecode suitable for Metal
118
119 \endlist
120
121 \sa QShaderBaker
122 */
123
124/*!
125 \enum QShader::Stage
126 Describes the stage of the graphics pipeline the shader is suitable for.
127
128 \value VertexStage Vertex shader
129 \value TessellationControlStage Tessellation control (hull) shader
130 \value TessellationEvaluationStage Tessellation evaluation (domain) shader
131 \value GeometryStage Geometry shader
132 \value FragmentStage Fragment (pixel) shader
133 \value ComputeStage Compute shader
134 */
135
136/*!
137 \class QShaderVersion
138 \internal
139 \inmodule QtGui
140
141 \brief Specifies the shading language version.
142
143 While languages like SPIR-V or the Metal Shading Language use traditional
144 version numbers, shaders for other APIs can use slightly different
145 versioning schemes. All those are mapped to a single version number in
146 here, however. For HLSL, the version refers to the Shader Model version,
147 like 5.0, 5.1, or 6.0. For GLSL an additional flag is needed to choose
148 between GLSL and GLSL/ES.
149
150 Below is a list with the most common examples of shader versions for
151 different graphics APIs:
152
153 \list
154
155 \li Vulkan (SPIR-V): 100
156 \li OpenGL: 120, 330, 440, etc.
157 \li OpenGL ES: 100 with GlslEs, 300 with GlslEs, etc.
158 \li Direct3D: 50, 51, 60
159 \li Metal: 12, 20
160 \endlist
161
162 A default constructed QShaderVersion contains a version of 100 and no
163 flags set.
164 */
165
166/*!
167 \enum QShaderVersion::Flag
168
169 Describes the flags that can be set.
170
171 \value GlslEs Indicates that GLSL/ES is meant in combination with GlslShader
172 */
173
174/*!
175 \class QShaderKey
176 \internal
177 \inmodule QtGui
178
179 \brief Specifies the shading language, the version with flags, and the variant.
180
181 A default constructed QShaderKey has source set to SpirvShader and
182 sourceVersion set to 100. sourceVariant defaults to StandardShader.
183 */
184
185/*!
186 \enum QShader::Source
187 Describes what kind of shader code an entry contains.
188
189 \value SpirvShader SPIR-V
190 \value GlslShader GLSL
191 \value HlslShader HLSL
192 \value DxbcShader Direct3D bytecode (HLSL compiled by \c fxc)
193 \value MslShader Metal Shading Language
194 \value DxilShader Direct3D bytecode (HLSL compiled by \c dxc)
195 \value MetalLibShader Pre-compiled Metal bytecode
196 */
197
198/*!
199 \enum QShader::Variant
200 Describes what kind of shader code an entry contains.
201
202 \value StandardShader The normal, unmodified version of the shader code.
203 \value BatchableVertexShader Vertex shader rewritten to be suitable for Qt Quick scenegraph batching.
204 */
205
206/*!
207 \class QShaderCode
208 \internal
209 \inmodule QtGui
210
211 \brief Contains source or binary code for a shader and additional metadata.
212
213 When shader() is empty after retrieving a QShaderCode instance from
214 QShader, it indicates no shader code was found for the requested key.
215 */
216
217/*!
218 Constructs a new, empty (and thus invalid) QShader instance.
219 */
220QShader::QShader()
221 : d(new QShaderPrivate)
222{
223}
224
225/*!
226 \internal
227 */
228void QShader::detach()
229{
230 qAtomicDetach(d);
231}
232
233/*!
234 \internal
235 */
236QShader::QShader(const QShader &other)
237 : d(other.d)
238{
239 d->ref.ref();
240}
241
242/*!
243 \internal
244 */
245QShader &QShader::operator=(const QShader &other)
246{
247 qAtomicAssign(d, other.d);
248 return *this;
249}
250
251/*!
252 Destructor.
253 */
254QShader::~QShader()
255{
256 if (!d->ref.deref())
257 delete d;
258}
259
260/*!
261 \return true if the QShader contains at least one shader version.
262 */
263bool QShader::isValid() const
264{
265 return !d->shaders.isEmpty();
266}
267
268/*!
269 \return the pipeline stage the shader is meant for.
270 */
271QShader::Stage QShader::stage() const
272{
273 return d->stage;
274}
275
276/*!
277 Sets the pipeline \a stage.
278 */
279void QShader::setStage(Stage stage)
280{
281 if (stage != d->stage) {
282 detach();
283 d->stage = stage;
284 }
285}
286
287/*!
288 \return the reflection metadata for the shader.
289 */
290QShaderDescription QShader::description() const
291{
292 return d->desc;
293}
294
295/*!
296 Sets the reflection metadata to \a desc.
297 */
298void QShader::setDescription(const QShaderDescription &desc)
299{
300 detach();
301 d->desc = desc;
302}
303
304/*!
305 \return the list of available shader versions
306 */
307QList<QShaderKey> QShader::availableShaders() const
308{
309 return d->shaders.keys().toVector();
310}
311
312/*!
313 \return the source or binary code for a given shader version specified by \a key.
314 */
315QShaderCode QShader::shader(const QShaderKey &key) const
316{
317 return d->shaders.value(key);
318}
319
320/*!
321 Stores the source or binary \a shader code for a given shader version specified by \a key.
322 */
323void QShader::setShader(const QShaderKey &key, const QShaderCode &shader)
324{
325 if (d->shaders.value(key) == shader)
326 return;
327
328 detach();
329 d->shaders[key] = shader;
330}
331
332/*!
333 Removes the source or binary shader code for a given \a key.
334 Does nothing when not found.
335 */
336void QShader::removeShader(const QShaderKey &key)
337{
338 auto it = d->shaders.find(key);
339 if (it == d->shaders.end())
340 return;
341
342 detach();
343 d->shaders.erase(it);
344}
345
346static void writeShaderKey(QDataStream *ds, const QShaderKey &k)
347{
348 *ds << int(k.source());
349 *ds << k.sourceVersion().version();
350 *ds << k.sourceVersion().flags();
351 *ds << int(k.sourceVariant());
352}
353
354/*!
355 \return a serialized binary version of all the data held by the
356 QShader, suitable for writing to files or other I/O devices.
357
358 \sa fromSerialized()
359 */
360QByteArray QShader::serialized() const
361{
362 QBuffer buf;
363 QDataStream ds(&buf);
364 ds.setVersion(QDataStream::Qt_5_10);
365 if (!buf.open(QIODevice::WriteOnly))
366 return QByteArray();
367
368 ds << QShaderPrivate::QSB_VERSION;
369 ds << int(d->stage);
370 d->desc.serialize(&ds);
371 ds << int(d->shaders.count());
372 for (auto it = d->shaders.cbegin(), itEnd = d->shaders.cend(); it != itEnd; ++it) {
373 const QShaderKey &k(it.key());
374 writeShaderKey(&ds, k);
375 const QShaderCode &shader(d->shaders.value(k));
376 ds << shader.shader();
377 ds << shader.entryPoint();
378 }
379 ds << int(d->bindings.count());
380 for (auto it = d->bindings.cbegin(), itEnd = d->bindings.cend(); it != itEnd; ++it) {
381 const QShaderKey &k(it.key());
382 writeShaderKey(&ds, k);
383 const NativeResourceBindingMap &map(it.value());
384 ds << int(map.count());
385 for (auto mapIt = map.cbegin(), mapItEnd = map.cend(); mapIt != mapItEnd; ++mapIt) {
386 ds << mapIt.key();
387 ds << mapIt.value().first;
388 ds << mapIt.value().second;
389 }
390 }
391
392 return qCompress(buf.buffer());
393}
394
395static void readShaderKey(QDataStream *ds, QShaderKey *k)
396{
397 int intVal;
398 *ds >> intVal;
399 k->setSource(QShader::Source(intVal));
400 QShaderVersion ver;
401 *ds >> intVal;
402 ver.setVersion(intVal);
403 *ds >> intVal;
404 ver.setFlags(QShaderVersion::Flags(intVal));
405 k->setSourceVersion(ver);
406 *ds >> intVal;
407 k->setSourceVariant(QShader::Variant(intVal));
408}
409
410/*!
411 Creates a new QShader instance from the given \a data.
412
413 \sa serialized()
414 */
415QShader QShader::fromSerialized(const QByteArray &data)
416{
417 QByteArray udata = qUncompress(data);
418 QBuffer buf(&udata);
419 QDataStream ds(&buf);
420 ds.setVersion(QDataStream::Qt_5_10);
421 if (!buf.open(QIODevice::ReadOnly))
422 return QShader();
423
424 QShader bs;
425 QShaderPrivate *d = QShaderPrivate::get(&bs);
426 Q_ASSERT(d->ref.loadRelaxed() == 1); // must be detached
427 int intVal;
428 ds >> intVal;
429 d->qsbVersion = intVal;
430 if (d->qsbVersion != QShaderPrivate::QSB_VERSION
431 && d->qsbVersion != QShaderPrivate::QSB_VERSION_WITHOUT_VAR_ARRAYDIMS
432 && d->qsbVersion != QShaderPrivate::QSB_VERSION_WITH_CBOR
433 && d->qsbVersion != QShaderPrivate::QSB_VERSION_WITH_BINARY_JSON
434 && d->qsbVersion != QShaderPrivate::QSB_VERSION_WITHOUT_BINDINGS)
435 {
436 qWarning("Attempted to deserialize QShader with unknown version %d.", d->qsbVersion);
437 return QShader();
438 }
439
440 ds >> intVal;
441 d->stage = Stage(intVal);
442 if (d->qsbVersion > QShaderPrivate::QSB_VERSION_WITH_CBOR) {
443 d->desc = QShaderDescription::deserialize(&ds, d->qsbVersion);
444 } else if (d->qsbVersion > QShaderPrivate::QSB_VERSION_WITH_BINARY_JSON) {
445 qWarning("Can no longer load QShaderDescription from CBOR.");
446 d->desc = QShaderDescription();
447 } else {
448 qWarning("Can no longer load QShaderDescription from binary JSON.");
449 d->desc = QShaderDescription();
450 }
451 int count;
452 ds >> count;
453 for (int i = 0; i < count; ++i) {
454 QShaderKey k;
455 readShaderKey(&ds, &k);
456 QShaderCode shader;
457 QByteArray s;
458 ds >> s;
459 shader.setShader(s);
460 ds >> s;
461 shader.setEntryPoint(s);
462 d->shaders[k] = shader;
463 }
464
465 if (d->qsbVersion > QShaderPrivate::QSB_VERSION_WITHOUT_BINDINGS) {
466 ds >> count;
467 for (int i = 0; i < count; ++i) {
468 QShaderKey k;
469 readShaderKey(&ds, &k);
470 NativeResourceBindingMap map;
471 int mapSize;
472 ds >> mapSize;
473 for (int b = 0; b < mapSize; ++b) {
474 int binding;
475 ds >> binding;
476 int firstNativeBinding;
477 ds >> firstNativeBinding;
478 int secondNativeBinding;
479 ds >> secondNativeBinding;
480 map.insert(binding, { firstNativeBinding, secondNativeBinding });
481 }
482 d->bindings.insert(k, map);
483 }
484 }
485
486 return bs;
487}
488
489QShaderVersion::QShaderVersion(int v, Flags f)
490 : m_version(v), m_flags(f)
491{
492}
493
494QShaderCode::QShaderCode(const QByteArray &code, const QByteArray &entry)
495 : m_shader(code), m_entryPoint(entry)
496{
497}
498
499QShaderKey::QShaderKey(QShader::Source s,
500 const QShaderVersion &sver,
501 QShader::Variant svar)
502 : m_source(s),
503 m_sourceVersion(sver),
504 m_sourceVariant(svar)
505{
506}
507
508/*!
509 Returns \c true if the two QShader objects \a lhs and \a rhs are equal,
510 meaning they are for the same stage with matching sets of shader source or
511 binary code.
512
513 \relates QShader
514 */
515bool operator==(const QShader &lhs, const QShader &rhs) noexcept
516{
517 return lhs.d->stage == rhs.d->stage
518 && lhs.d->shaders == rhs.d->shaders;
519 // do not bother with desc and bindings, if the shader code is the same, the description must match too
520}
521
522/*!
523 \internal
524 \fn bool operator!=(const QShader &lhs, const QShader &rhs)
525
526 Returns \c false if the values in the two QShader objects \a a and \a b
527 are equal; otherwise returns \c true.
528
529 \relates QShader
530 */
531
532/*!
533 Returns the hash value for \a s, using \a seed to seed the calculation.
534
535 \relates QShader
536 */
537size_t qHash(const QShader &s, size_t seed) noexcept
538{
539 QtPrivate::QHashCombine hash;
540 seed = hash(seed, s.stage());
541 seed = qHashRange(s.d->shaders.keyValueBegin(),
542 s.d->shaders.keyValueEnd(),
543 seed);
544 return seed;
545}
546
547/*!
548 Returns \c true if the two QShaderVersion objects \a lhs and \a rhs are
549 equal.
550
551 \relates QShaderVersion
552 */
553bool operator==(const QShaderVersion &lhs, const QShaderVersion &rhs) noexcept
554{
555 return lhs.version() == rhs.version() && lhs.flags() == rhs.flags();
556}
557
558/*!
559 \internal
560 \fn bool operator!=(const QShaderVersion &lhs, const QShaderVersion &rhs)
561
562 Returns \c false if the values in the two QShaderVersion objects \a a
563 and \a b are equal; otherwise returns \c true.
564
565 \relates QShaderVersion
566 */
567
568/*!
569 Returns \c true if the two QShaderKey objects \a lhs and \a rhs are equal.
570
571 \relates QShaderKey
572 */
573bool operator==(const QShaderKey &lhs, const QShaderKey &rhs) noexcept
574{
575 return lhs.source() == rhs.source() && lhs.sourceVersion() == rhs.sourceVersion()
576 && lhs.sourceVariant() == rhs.sourceVariant();
577}
578
579/*!
580 \internal
581 \fn bool operator!=(const QShaderKey &lhs, const QShaderKey &rhs)
582
583 Returns \c false if the values in the two QShaderKey objects \a a
584 and \a b are equal; otherwise returns \c true.
585
586 \relates QShaderKey
587 */
588
589/*!
590 Returns the hash value for \a k, using \a seed to seed the calculation.
591
592 \relates QShaderKey
593 */
594size_t qHash(const QShaderKey &k, size_t seed) noexcept
595{
596 return qHashMulti(seed,
597 k.source(),
598 k.sourceVersion().version(),
599 k.sourceVersion().flags(),
600 k.sourceVariant());
601}
602
603/*!
604 Returns \c true if the two QShaderCode objects \a lhs and \a rhs are equal.
605
606 \relates QShaderCode
607 */
608bool operator==(const QShaderCode &lhs, const QShaderCode &rhs) noexcept
609{
610 return lhs.shader() == rhs.shader() && lhs.entryPoint() == rhs.entryPoint();
611}
612
613/*!
614 \internal
615 \fn bool operator!=(const QShaderCode &lhs, const QShaderCode &rhs)
616
617 Returns \c false if the values in the two QShaderCode objects \a a
618 and \a b are equal; otherwise returns \c true.
619
620 \relates QShaderCode
621 */
622
623/*!
624 Returns the hash value for \a k, using \a seed to seed the calculation.
625
626 \relates QShaderCode
627 */
628size_t qHash(const QShaderCode &k, size_t seed) noexcept
629{
630 return qHash(k.shader(), seed);
631}
632
633#ifndef QT_NO_DEBUG_STREAM
634QDebug operator<<(QDebug dbg, const QShader &bs)
635{
636 const QShaderPrivate *d = bs.d;
637 QDebugStateSaver saver(dbg);
638
639 dbg.nospace() << "QShader("
640 << "stage=" << d->stage
641 << " shaders=" << d->shaders.keys()
642 << " desc.isValid=" << d->desc.isValid()
643 << ')';
644
645 return dbg;
646}
647
648QDebug operator<<(QDebug dbg, const QShaderKey &k)
649{
650 QDebugStateSaver saver(dbg);
651 dbg.nospace() << "ShaderKey(" << k.source()
652 << " " << k.sourceVersion()
653 << " " << k.sourceVariant() << ")";
654 return dbg;
655}
656
657QDebug operator<<(QDebug dbg, const QShaderVersion &v)
658{
659 QDebugStateSaver saver(dbg);
660 dbg.nospace() << "Version(" << v.version() << " " << v.flags() << ")";
661 return dbg;
662}
663#endif // QT_NO_DEBUG_STREAM
664
665/*!
666 \typedef QShader::NativeResourceBindingMap
667
668 Synonym for QHash<int, QPair<int, int>>.
669
670 The resource binding model QRhi assumes is based on SPIR-V. This means that
671 uniform buffers, storage buffers, combined image samplers, and storage
672 images share a common binding point space. The binding numbers in
673 QShaderDescription and QRhiShaderResourceBinding are expected to match the
674 \c binding layout qualifier in the Vulkan-compatible GLSL shader.
675
676 Graphics APIs other than Vulkan may use a resource binding model that is
677 not fully compatible with this. In addition, the generator of the shader
678 code translated from SPIR-V may choose not to take the SPIR-V binding
679 qualifiers into account, for various reasons. (this is the case with the
680 Metal backend of SPIRV-Cross, for example).
681
682 Therefore, a QShader may expose an additional map that describes what the
683 native binding point for a given SPIR-V binding is. The QRhi backends are
684 expected to use this map automatically, as appropriate. The value is a
685 pair, because combined image samplers may map to two native resources (a
686 texture and a sampler) in some shading languages. In that case the second
687 value refers to the sampler.
688
689 \note The native binding may be -1, in case there is no active binding for
690 the resource in the shader. (for example, there is a uniform block
691 declared, but it is not used in the shader code) The map is always
692 complete, meaning there is an entry for all declared uniform blocks,
693 storage blocks, image objects, and combined samplers, but the value will be
694 -1 for those that are not actually referenced in the shader functions.
695*/
696
697/*!
698 \return the native binding map for \a key or null if no extra mapping is
699 available, or is not applicable.
700 */
701const QShader::NativeResourceBindingMap *QShader::nativeResourceBindingMap(const QShaderKey &key) const
702{
703 auto it = d->bindings.constFind(key);
704 if (it == d->bindings.cend())
705 return nullptr;
706
707 return &it.value();
708}
709
710/*!
711 Stores the given native resource binding \a map associated with \a key.
712
713 \sa nativeResourceBindingMap()
714 */
715void QShader::setResourceBindingMap(const QShaderKey &key, const NativeResourceBindingMap &map)
716{
717 detach();
718 d->bindings[key] = map;
719}
720
721/*!
722 Removes the native resource binding map for \a key.
723 */
724void QShader::removeResourceBindingMap(const QShaderKey &key)
725{
726 auto it = d->bindings.find(key);
727 if (it == d->bindings.end())
728 return;
729
730 detach();
731 d->bindings.erase(it);
732}
733
734QT_END_NAMESPACE
735