Meshletize - checklist vector<bool> subscript out of range

[vivi168]

For some mesh, the checklist vector is not properly sized, thus we get Expression: vector<bool> subscript out of range originating at this line:

DirectXMesh/DirectXMesh/DirectXMeshletGenerator.cpp

Line 362 in 5be3c74

if (checklist[adj[i] - startIndex])

In my case, we are short of just one index, (adj[i] - startIndex) == checklist.size()

So as a quick workaround I added +1 to the resize at L261

checklist.resize(subset.second + 1);

I can reproduce this error with BrainStem.gltf available here

After loading the GLTF file in a struct like this:

struct Mesh {
  std::vector<UINT> indices;
  std::vector<Primitive> primitives;

  std::vector<XMFLOAT3> positions;
  std::vector<XMFLOAT3> normals;
  std::vector<XMFLOAT2> uvs;
}

I'm using ComputeMeshlets with subsets.

std::vector<MeshletSubset> subsets;
subsets.reserve(primitives.size());

for (const auto& prim : primitives) {
  subsets.emplace_back(prim.start / 3, prim.count / 3);
}

std::vector<MeshletSubset> meshletSubsets(primitives.size());

constexpr size_t maxPrims = 126;
constexpr size_t maxVerts = 64;
CHECK_HR(ComputeMeshlets(
    indices.data(), indices.size() / 3,
    positions.data(), positions.size(),
    subsets.data(), subsets.size(),
    nullptr,
    meshlets, uniqueVertexIndices, primitiveIndices,
    meshletSubsets.data(),
    maxVerts, maxPrims));

[walbourn]

Subsets are not the same as faces. They are intended to group materials. The code above looks like you are trying to create one subset per face. Is that the case?

[vivi168]

Sorry, I realize that the variable names in the snippet above are confusing

primitives is indeed subsets (it's the GLTF term, and its count and start members are offsets into mesh indices[], yet ComputeMeshlet expects subsets with face offsets hence the divide by 3)

for example the BrainStem mesh has 49 GLTF primitives (subsets)

[vivi168]

Shouldn't the following guard condition be adj[i] >= endIndex ?

DirectXMesh/DirectXMesh/DirectXMeshletGenerator.cpp

Line 358 in 5be3c74

if (adj[i] < subset.first || adj[i] > endIndex)

// Seed the candidate list with the first triangle of the subset
const uint32_t startIndex = static_cast<uint32_t>(subset.first);
const uint32_t endIndex = static_cast<uint32_t>(subset.first + subset.second);

let's say we have a subset { .start = 0, .count = 5 }
endIndex is then 5.
surely, if adj[i] == 5, it's outside the subset. (0, 1, 2, 3, 4 are in)

[vivi168]

Hi @walbourn

Could I get your feedback on the above comment and linked PR?
Thank you!