RuntimeError: CUDA is available for PyTorch, but NNcF could not compile GPu quan tization extensions.

[huangqiu15444]

🐛 Describe the bug

nccfv2.5.0 and torch1.9.1 can work normally on the PC, but do not work on the Linux system。
Are there any methods to quickly troubleshoot the issue? Many colleagues have also encountered similar problems.

Environment

nccf v2.5.0 + torch1.9.1
PC cuda env : CUDA10.2
Linux cuda env: CUDA11.1

Minimal Reproducible Example

quantized_model = nncf.quantize(net, calibration_dataset)

Are you going to submit a PR?

• Yes I'd like to help by submitting a PR!

[alexsu52]

Hello @huangqiu15444, Could you provide the log?

Try removing "~/.cache/torch_extensions/", this often helps.

[huangqiu15444]

Hello @huangqiu15444, Could you provide the log?

Try removing "~/.cache/torch_extensions/", this often helps.

Here is the log.
Traceback (most recent call last):
File "/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 1666, in _run_ninja_build
subprocess.run(
File "/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/subprocess.py", line 516, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['ninja', '-v']' returned non-zero exit status 1.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/nncf/torch/quantization/extensions.py", line 83, in load
return torch.utils.cpp_extension.load(
File "/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 1080, in load
return jit_compile(
File "/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 1293, in jit_compile
write_ninja_file_and_build_library(
File "/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 1405, in write_ninja_file_and_build_library
run_ninja_build(
File "/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 1682, in run_ninja_build
raise RuntimeError(message) from e
RuntimeError: Error building extension 'quantized_functions_cuda': [1/3] /usr/bin/nvcc -DTORCH_EXTENSION_NAME=quantized_functions_cuda -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE="gcc" -DPYBIND11_STDLIB="libstdcpp" -DPYBIND11_BUILD_ABI="cxxabi1011" -I/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/nncf/torch/extensions/include -isystem /home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/torch/include -isystem /home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/torch/include/torch/csrc/api/include -isystem /home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/torch/include/TH -isystem /home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/torch/include/THC -isystem /home/chenxc/anaconda3/envs/Grounding/include/python3.8 -D_GLIBCXX_USE_CXX11_ABI=0 -D__CUDA_NO_HALF_OPERATORS -D__CUDA_NO_HALF_CONVERSIONS -D__CUDA_NO_BFLOAT16_CONVERSIONS -D__CUDA_NO_HALF2_OPERATORS --expt-relaxed-constexpr -gencode=arch=compute_86,code=compute_86 -gencode=arch=compute_86,code=sm_86 --compiler-options '-fPIC' -std=c++14 -c /home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/nncf/torch/extensions/src/quantization/cuda/functions_cuda_impl.cu -o functions_cuda_impl.cuda.o
FAILED: functions_cuda_impl.cuda.o
/usr/bin/nvcc -DTORCH_EXTENSION_NAME=quantized_functions_cuda -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE="gcc" -DPYBIND11_STDLIB="libstdcpp" -DPYBIND11_BUILD_ABI="cxxabi1011" -I/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/nncf/torch/extensions/include -isystem /home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/torch/include -isystem /home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/torch/include/torch/csrc/api/include -isystem /home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/torch/include/TH -isystem /home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/torch/include/THC -isystem /home/chenxc/anaconda3/envs/Grounding/include/python3.8 -D_GLIBCXX_USE_CXX11_ABI=0 -D__CUDA_NO_HALF_OPERATORS -D__CUDA_NO_HALF_CONVERSIONS -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr -gencode=arch=compute_86,code=compute_86 -gencode=arch=compute_86,code=sm_86 --compiler-options '-fPIC' -std=c++14 -c /home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/nncf/torch/extensions/src/quantization/cuda/functions_cuda_impl.cu -o functions_cuda_impl.cuda.o
/usr/include/c++/11/type_traits(1406): error: type name is not allowed

/usr/include/c++/11/type_traits(1406): error: type name is not allowed

/usr/include/c++/11/type_traits(1406): error: identifier "__is_same" is undefined

/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/torch/include/c10/util/Metaprogramming.h(425): error: no instance of constructor "std::tuple<_T1, _T2>::tuple [with _T1=int32_t &&, _T2=int32_t]" matches the argument list
argument types are: (int32_t, int)

/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/torch/include/c10/util/Metaprogramming.h(423): error: no instance of overloaded function "c10::guts::detail::extract_tuple_element_by_index" matches the argument list
argument types are: (std::tuple<int32_t>, )

/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/torch/include/c10/core/Scalar.h(32): error: no instance of constructor "c10::Scalar::Scalar" matches the argument list
argument types are: (uint8_t, __nv_bool)

/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/torch/include/c10/core/Scalar.h(32): error: no instance of constructor "c10::Scalar::Scalar" matches the argument list
argument types are: (int8_t, __nv_bool)

/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/torch/include/c10/core/Scalar.h(32): error: no instance of constructor "c10::Scalar::Scalar" matches the argument list
argument types are: (int16_t, __nv_bool)

/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/torch/include/c10/core/Scalar.h(32): error: no instance of constructor "c10::Scalar::Scalar" matches the argument list
argument types are: (int, __nv_bool)

/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/torch/include/c10/core/Scalar.h(32): error: no instance of constructor "c10::Scalar::Scalar" matches the argument list
argument types are: (int64_t, __nv_bool)

/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/torch/include/c10/core/ScalarTypeToTypeMeta.h(34): error: no suitable constructor exists to convert from "c10::ScalarType" to "c10::optionalc10::ScalarType"

/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/torch/include/c10/core/TensorOptions.h(148): error: no suitable constructor exists to convert from "c10::Layout" to "c10::optionalc10::Layout"

/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/torch/include/c10/core/TensorOptions.h(177): error: no instance of overloaded function "c10::TensorOptions::set_dtype" matches the argument list
argument types are: (caffe2::TypeMeta)
object type is: c10::TensorOptions

/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/torch/include/c10/core/TensorOptions.h(182): error: no instance of overloaded function "c10::TensorOptions::set_dtype" matches the argument list
argument types are: (c10::ScalarType)
object type is: c10::TensorOptions

/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/torch/include/c10/core/TensorOptions.h(187): error: no suitable constructor exists to convert from "c10::MemoryFormat" to "c10::optionalc10::MemoryFormat"

/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/torch/include/c10/util/Optional.h(1116): error: no instance of constructor "c10::optional::optional [with T=c10::Device]" matches the argument list
argument types are: (const c10::Device)
detected during instantiation of "c10::optional<std::decay::type> c10::make_optional(T &&) [with T=const c10::Device &]"
/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/torch/include/c10/core/TensorOptions.h(287): here

/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/torch/include/c10/util/Optional.h(1116): error: no instance of constructor "c10::optional::optional [with T=caffe2::TypeMeta]" matches the argument list
argument types are: (const caffe2::TypeMeta)
detected during instantiation of "c10::optional<std::decay::type> c10::make_optional(T &&) [with T=const caffe2::TypeMeta &]"
/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/torch/include/c10/core/TensorOptions.h(308): here

/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/torch/include/c10/util/Optional.h(1116): error: no instance of constructor "c10::optional::optional [with T=c10::Layout]" matches the argument list
argument types are: (const c10::Layout)
detected during instantiation of "c10::optional<std::decay::type> c10::make_optional(T &&) [with T=const c10::Layout &]"
/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/torch/include/c10/core/TensorOptions.h(324): here

/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/torch/include/c10/util/Optional.h(1116): error: no instance of constructor "c10::optional::optional [with T=__nv_bool]" matches the argument list
argument types are: (const __nv_bool)
detected during instantiation of "c10::optional<std::decay::type> c10::make_optional(T &&) [with T=const __nv_bool &]"
/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/torch/include/c10/core/TensorOptions.h(341): here

/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/torch/include/c10/util/Optional.h(1116): error: no instance of constructor "c10::optional::optional [with T=c10::MemoryFormat]" matches the argument list
argument types are: (const c10::MemoryFormat)
detected during instantiation of "c10::optional<std::decay::type> c10::make_optional(T &&) [with T=const c10::MemoryFormat &]"
/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/torch/include/c10/core/TensorOptions.h(388): here

/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/torch/include/c10/core/TensorOptions.h(430): error: no suitable constructor exists to convert from "c10::MemoryFormat" to "c10::optionalc10::MemoryFormat"

/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/torch/include/c10/core/TensorOptions.h(568): error: no instance of overloaded function "c10::TensorOptions::dtype" matches the argument list
argument types are: (caffe2::TypeMeta)
object type is: c10::TensorOptions

/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/torch/include/c10/core/TensorOptions.h(573): error: no instance of overloaded function "c10::TensorOptions::dtype" matches the argument list
argument types are: (caffe2::TypeMeta)
object type is: c10::TensorOptions

/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/torch/include/c10/core/TensorOptions.h(579): error: no instance of overloaded function "c10::TensorOptions::layout" matches the argument list
argument types are: (c10::Layout)
object type is: c10::TensorOptions

/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/torch/include/c10/core/TensorOptions.h(597): error: no instance of overloaded function "c10::TensorOptions::requires_grad" matches the argument list
argument types are: (__nv_bool)
object type is: c10::TensorOptions

/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/torch/include/c10/core/TensorOptions.h(603): error: no suitable constructor exists to convert from "c10::MemoryFormat" to "c10::optionalc10::MemoryFormat"

/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/torch/include/c10/core/TensorOptions.h(795): error: no instance of overloaded function "c10::TensorOptions::layout" matches the argument list
argument types are: (c10::Layout)
object type is: c10::TensorOptions

/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/torch/include/c10/util/Optional.h(585): error: class "std::enable_if<, c10::optionalc10::Device &>" has no member "type"
detected during instantiation of "c10::optional::operator= [with T=c10::Device]" based on template argument c10::Device
/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/torch/include/c10/core/TensorImpl.h(1562): here

/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/torch/include/ATen/core/DeprecatedTypeProperties.h(99): error: no instance of overloaded function "c10::TensorOptions::dtype" matches the argument list
argument types are: (caffe2::TypeMeta)
object type is: c10::TensorOptions

/usr/include/c++/11/type_traits(2579): error: class "std::enable_if<, c10::complex>" has no member "type"
detected during:
instantiation of type "std::enable_if_t<, c10::complex>"
/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/torch/include/c10/util/complex.h(169): here
instantiation of "c10::complex::complex [with T=double]" based on template argument
(946): here
instantiation of class "std::__is_constructible_impl<_Tp, _Args...> [with _Tp=c10::Scalar, _Args=<const c10::nullopt_t &>]"
(953): here
instantiation of class "std::is_constructible<_Tp, _Args...> [with _Tp=c10::Scalar, _Args=<const c10::nullopt_t &>]"
/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/torch/include/ATen/core/TensorBody.h(828): here

/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/torch/include/ATen/core/TensorBody.h(668): error: no suitable user-defined conversion from "const at::Tensor" to "const c10::optionalat::Tensor" exists

/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/torch/include/ATen/core/TensorBody.h(670): error: no suitable user-defined conversion from "const at::Tensor" to "const c10::optionalat::Tensor" exists

/usr/include/c++/11/type_traits(1406): error: type name is not allowed
detected during instantiation of class "std::is_same<_Tp, _Up> [with _Tp=at::Tensor, _Up=__nv_bool]"
/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/torch/include/ATen/core/TensorBody.h(1670): here

/usr/include/c++/11/type_traits(1406): error: type name is not allowed
detected during instantiation of class "std::is_same<_Tp, _Up> [with _Tp=at::Tensor, _Up=__nv_bool]"
/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/torch/include/ATen/core/TensorBody.h(1670): here

/usr/include/c++/11/type_traits(1406): error: type name is not allowed
detected during instantiation of class "std::is_same<_Tp, _Up> [with _Tp=c10::complex, _Up=c10::Device]"
/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/torch/include/ATen/core/Formatting.h(30): here

/usr/include/c++/11/type_traits(1406): error: type name is not allowed
detected during instantiation of class "std::is_same<_Tp, _Up> [with _Tp=c10::complex, _Up=c10::Device]"
/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/torch/include/ATen/core/Formatting.h(30): here

/usr/include/c++/11/type_traits(1406): error: type name is not allowed
detected during instantiation of class "std::is_same<_Tp, _Up> [with _Tp=const char *, _Up=__nv_bool]"
/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/torch/include/ATen/core/Formatting.h(33): here

/usr/include/c++/11/type_traits(1406): error: type name is not allowed
detected during instantiation of class "std::is_same<_Tp, _Up> [with _Tp=const char *, _Up=__nv_bool]"
/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/torch/include/ATen/core/Formatting.h(33): here

/usr/include/c++/11/type_traits(1406): error: type name is not allowed
detected during:
instantiation of class "std::is_same<_Tp, _Up> [with _Tp=c10::TensorImpl *, _Up=c10::TensorImpl *]"
/usr/include/c++/11/bits/stl_vector.h(401): here
instantiation of class "std::vector<_Tp, _Alloc> [with _Tp=c10::TensorImpl *, _Alloc=std::allocator<c10::TensorImpl *>]"
/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/torch/include/ATen/Utils.h(52): here

/usr/include/c++/11/type_traits(1406): error: type name is not allowed
detected during:
instantiation of class "std::is_same<_Tp, _Up> [with _Tp=c10::TensorImpl *, _Up=c10::TensorImpl *]"
/usr/include/c++/11/bits/stl_vector.h(401): here
instantiation of class "std::vector<_Tp, _Alloc> [with _Tp=c10::TensorImpl *, _Alloc=std::allocator<c10::TensorImpl *>]"
/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/torch/include/ATen/Utils.h(52): here

/usr/include/c++/11/type_traits(1406): error: type name is not allowed
detected during:
instantiation of class "std::is_same<_Tp, _Up> [with _Tp=float, _Up=c10::nullopt_t]"
/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/torch/include/c10/util/Optional.h(455): here
instantiation of class "c10::optional [with T=float]"
/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/torch/include/ATen/CPUGeneratorImpl.h(35): here

/usr/include/c++/11/type_traits(1406): error: type name is not allowed
detected during:
instantiation of class "std::is_same<_Tp, _Up> [with _Tp=float, _Up=c10::nullopt_t]"
/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/torch/include/c10/util/Optional.h(455): here
instantiation of class "c10::optional [with T=float]"
/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/torch/include/ATen/CPUGeneratorImpl.h(35): here

/usr/include/c++/11/type_traits(1406): error: type name is not allowed
detected during:
instantiation of class "std::is_same<_Tp, _Up> [with _Tp=float, _Up=c10::in_place_t]"
/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/torch/include/c10/util/Optional.h(458): here
instantiation of class "c10::optional [with T=float]"
/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/torch/include/ATen/CPUGeneratorImpl.h(35): here

/usr/include/c++/11/type_traits(1406): error: type name is not allowed
detected during:
instantiation of class "std::is_same<_Tp, _Up> [with _Tp=float, _Up=c10::in_place_t]"
/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/torch/include/c10/util/Optional.h(458): here
instantiation of class "c10::optional [with T=float]"
/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/torch/include/ATen/CPUGeneratorImpl.h(35): here

/usr/include/c++/11/type_traits(1406): error: type name is not allowed
detected during:
instantiation of class "std::is_same<_Tp, _Up> [with _Tp=c10::QEngine, _Up=c10::nullopt_t]"
/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/torch/include/c10/util/Optional.h(455): here
instantiation of class "c10::optional [with T=c10::QEngine]"
/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/torch/include/ATen/Context.h(241): here

/usr/include/c++/11/type_traits(1406): error: type name is not allowed
detected during:
instantiation of class "std::is_same<_Tp, _Up> [with _Tp=c10::QEngine, _Up=c10::nullopt_t]"
/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/torch/include/c10/util/Optional.h(455): here
instantiation of class "c10::optional [with T=c10::QEngine]"
/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/torch/include/ATen/Context.h(241): here

/usr/include/c++/11/type_traits(1406): error: type name is not allowed
detected during:
instantiation of class "std::is_same<_Tp, _Up> [with _Tp=c10::QEngine, _Up=c10::in_place_t]"
/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/torch/include/c10/util/Optional.h(458): here
instantiation of class "c10::optional [with T=c10::QEngine]"
/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/torch/include/ATen/Context.h(241): here

/usr/include/c++/11/type_traits(1406): error: type name is not allowed
detected during:
instantiation of class "std::is_same<_Tp, _Up> [with _Tp=c10::QEngine, _Up=c10::in_place_t]"
/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/torch/include/c10/util/Optional.h(458): here
instantiation of class "c10::optional [with T=c10::QEngine]"
/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/torch/include/ATen/Context.h(241): here

/usr/include/c++/11/type_traits(1406): error: type name is not allowed
detected during instantiation of class "std::is_same<_Tp, _Up> [with _Tp=c10::nullopt_t, _Up=c10::optionalc10::QEngine]"
/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/torch/include/ATen/Context.h(241): here

/usr/include/c++/11/type_traits(1406): error: type name is not allowed
detected during instantiation of class "std::is_same<_Tp, _Up> [with _Tp=c10::nullopt_t, _Up=c10::optionalc10::QEngine]"
/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/torch/include/ATen/Context.h(241): here

/usr/include/c++/11/type_traits(1406): error: type name is not allowed
detected during instantiation of class "std::is_same<_Tp, _Up> [with _Tp=c10::impl::VirtualGuardImpl, _Up=c10::impl::VirtualGuardImpl]"
/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/torch/include/c10/core/DeviceGuard.h(31): here

/usr/include/c++/11/type_traits(1406): error: type name is not allowed
detected during instantiation of class "std::is_same<_Tp, _Up> [with _Tp=c10::impl::VirtualGuardImpl, _Up=c10::impl::VirtualGuardImpl]"
/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/torch/include/c10/core/DeviceGuard.h(31): here

/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/torch/include/c10/core/DeviceGuard.h(31): error: no instance of constructor "c10::impl::InlineDeviceGuard::InlineDeviceGuard [with T=c10::impl::VirtualGuardImpl]" matches the argument list
argument types are: (c10::Device, const c10::impl::DeviceGuardImplInterface *)

/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/torch/include/c10/core/DeviceGuard.h(50): error: more than one instance of overloaded function "c10::impl::InlineDeviceGuard::reset_device [with T=c10::impl::VirtualGuardImpl]" matches the argument list:
function template "std::enable_if<, void>::type c10::impl::InlineDeviceGuard::reset_device(c10::Device) [with T=c10::impl::VirtualGuardImpl]"
/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/torch/include/c10/core/impl/InlineDeviceGuard.h(136): here
function template "std::enable_if<std::is_same<U, c10::impl::VirtualGuardImpl>::value, void>::type c10::impl::InlineDeviceGuard::reset_device(c10::Device, const c10::impl::DeviceGuardImplInterface *) [with T=c10::impl::VirtualGuardImpl]"
/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/torch/include/c10/core/impl/InlineDeviceGuard.h(159): here
argument types are: (c10::Device)
object type is: c10::impl::InlineDeviceGuardc10::impl::VirtualGuardImpl

/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/torch/include/c10/core/impl/InlineDeviceGuard.h(158): error: class "std::enable_if<, void>" has no member "type"
detected during instantiation of "c10::impl::InlineDeviceGuard::reset_device [with T=c10::impl::VirtualGuardImpl]" based on template argument c10::impl::VirtualGuardImpl
/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/torch/include/c10/core/DeviceGuard.h(57): here

/usr/include/c++/11/type_traits(1406): error: type name is not allowed
detected during:
instantiation of class "std::is_same<_Tp, _Up> [with _Tp=c10::impl::InlineDeviceGuardc10::impl::VirtualGuardImpl, _Up=c10::nullopt_t]"
/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/torch/include/c10/util/Optional.h(455): here
instantiation of class "c10::optional [with T=c10::impl::InlineDeviceGuardc10::impl::VirtualGuardImpl]"
/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/torch/include/c10/core/impl/InlineDeviceGuard.h(427): here
instantiation of class "c10::impl::InlineOptionalDeviceGuard [with T=c10::impl::VirtualGuardImpl]"
/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/torch/include/c10/core/DeviceGuard.h(178): here

/usr/include/c++/11/type_traits(1406): error: type name is not allowed
detected during:
instantiation of class "std::is_same<_Tp, _Up> [with _Tp=c10::impl::InlineDeviceGuardc10::impl::VirtualGuardImpl, _Up=c10::nullopt_t]"
/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/torch/include/c10/util/Optional.h(455): here
instantiation of class "c10::optional [with T=c10::impl::InlineDeviceGuardc10::impl::VirtualGuardImpl]"
/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/torch/include/c10/core/impl/InlineDeviceGuard.h(427): here
instantiation of class "c10::impl::InlineOptionalDeviceGuard [with T=c10::impl::VirtualGuardImpl]"
/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/torch/include/c10/core/DeviceGuard.h(178): here

/usr/include/c++/11/type_traits(1406): error: type name is not allowed
detected during:
instantiation of class "std::is_same<_Tp, _Up> [with _Tp=c10::impl::InlineDeviceGuardc10::impl::VirtualGuardImpl, _Up=c10::in_place_t]"
/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/torch/include/c10/util/Optional.h(458): here
instantiation of class "c10::optional [with T=c10::impl::InlineDeviceGuardc10::impl::VirtualGuardImpl]"
/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/torch/include/c10/core/impl/InlineDeviceGuard.h(427): here
instantiation of class "c10::impl::InlineOptionalDeviceGuard [with T=c10::impl::VirtualGuardImpl]"
/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/torch/include/c10/core/DeviceGuard.h(178): here

/usr/include/c++/11/type_traits(1406): error: type name is not allowed
detected during:
instantiation of class "std::is_same<_Tp, _Up> [with _Tp=c10::impl::InlineDeviceGuardc10::impl::VirtualGuardImpl, _Up=c10::in_place_t]"
/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/torch/include/c10/util/Optional.h(458): here
instantiation of class "c10::optional [with T=c10::impl::InlineDeviceGuardc10::impl::VirtualGuardImpl]"
/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/torch/include/c10/core/impl/InlineDeviceGuard.h(427): here
instantiation of class "c10::impl::InlineOptionalDeviceGuard [with T=c10::impl::VirtualGuardImpl]"
/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/torch/include/c10/core/DeviceGuard.h(178): here

/usr/include/c++/11/type_traits(1406): error: type name is not allowed
detected during:
instantiation of class "std::is_same<_Tp, _Up> [with _Tp=int8_t, _Up=c10::nullopt_t]"
/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/torch/include/c10/util/Optional.h(455): here
instantiation of class "c10::optional [with T=c10::DeviceIndex]"
/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/torch/include/c10/core/DeviceGuard.h(130): here

/usr/include/c++/11/type_traits(1406): error: type name is not allowed
detected during:
instantiation of class "std::is_same<_Tp, _Up> [with _Tp=int8_t, _Up=c10::nullopt_t]"
/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/torch/include/c10/util/Optional.h(455): here
instantiation of class "c10::optional [with T=c10::DeviceIndex]"
/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/torch/include/c10/core/DeviceGuard.h(130): here

/usr/include/c++/11/type_traits(1406): error: type name is not allowed
detected during instantiation of class "std::is_same<_Tp, _Up> [with _Tp=c10::Device, _Up=c10::optionalc10::DeviceIndex]"
/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/torch/include/c10/core/DeviceGuard.h(130): here

/usr/include/c++/11/type_traits(1406): error: type name is not allowed
detected during instantiation of class "std::is_same<_Tp, _Up> [with _Tp=c10::Device, _Up=c10::optionalc10::DeviceIndex]"
/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/torch/include/c10/core/DeviceGuard.h(130): here

/usr/include/c++/11/type_traits(1406): error: type name is not allowed
detected during instantiation of class "std::is_same<_Tp, _Up> [with _Tp=c10::optionalc10::Device, _Up=c10::optionalc10::DeviceIndex]"
/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/torch/include/c10/core/DeviceGuard.h(134): here

/usr/include/c++/11/type_traits(1406): error: type name is not allowed
detected during instantiation of class "std::is_same<_Tp, _Up> [with _Tp=c10::optionalc10::Device, _Up=c10::optionalc10::DeviceIndex]"
/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/torch/include/c10/core/DeviceGuard.h(134): here

/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/torch/include/c10/core/DeviceGuard.h(156): error: no instance of overloaded function "c10::impl::InlineOptionalDeviceGuard::reset_device [with T=c10::impl::VirtualGuardImpl]" matches the argument list
argument types are: (c10::Device)
object type is: c10::impl::InlineOptionalDeviceGuardc10::impl::VirtualGuardImpl

/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/torch/include/c10/core/DeviceGuard.h(163): error: no instance of overloaded function "c10::impl::InlineOptionalDeviceGuard::reset_device [with T=c10::impl::VirtualGuardImpl]" matches the argument list
argument types are: (c10::Device, const c10::impl::DeviceGuardImplInterface *)
object type is: c10::impl::InlineOptionalDeviceGuardc10::impl::VirtualGuardImpl

/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/torch/include/c10/util/Optional.h(1116): error: no instance of constructor "c10::optional::optional [with T=c10::Device]" matches the argument list
argument types are: (c10::Device)
detected during instantiation of "c10::optional<std::decay::type> c10::make_optional(T &&) [with T=c10::Device]"
/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/torch/include/ATen/DeviceGuard.h(19): here

/usr/include/c++/11/type_traits(1406): error: type name is not allowed
detected during instantiation of class "std::is_same<_Tp, _Up> [with _Tp=c10::string_view, _Up=c10::Device]"
/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/torch/include/ATen/core/blob.h(206): here

/usr/include/c++/11/type_traits(1406): error: type name is not allowed
detected during instantiation of class "std::is_same<_Tp, _Up> [with _Tp=c10::string_view, _Up=c10::Device]"
/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/torch/include/ATen/core/blob.h(206): here

/usr/include/c++/11/type_traits(1406): error: type name is not allowed
detected during instantiation of class "std::is_same<_Tp, _Up> [with _Tp=c10::IValue, _Up=__nv_bool]"
/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/torch/include/ATen/core/ivalue.h(208): here

/usr/include/c++/11/type_traits(1406): error: type name is not allowed
detected during instantiation of class "std::is_same<_Tp, _Up> [with _Tp=c10::IValue, _Up=__nv_bool]"
/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/torch/include/ATen/core/ivalue.h(208): here

/usr/include/c++/11/type_traits(1406): error: type name is not allowed
detected during instantiation of class "std::is_same<_Tp, _Up> [with _Tp=std::string, _Up=__nv_bool]"
/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/torch/include/ATen/core/ivalue.h(557): here

/usr/include/c++/11/type_traits(1406): error: type name is not allowed
detected during instantiation of class "std::is_same<_Tp, _Up> [with _Tp=std::string, _Up=__nv_bool]"
/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/torch/include/ATen/core/ivalue.h(557): here

/usr/include/c++/11/type_traits(1406): error: type name is not allowed
detected during instantiation of class "std::is_same<_Tp, _Up> [with _Tp=__nv_bool, _Up=__nv_bool]"
/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/torch/include/ATen/core/ivalue.h(711): here

/usr/include/c++/11/type_traits(1406): error: type name is not allowed
detected during instantiation of class "std::is_same<_Tp, _Up> [with _Tp=__nv_bool, _Up=__nv_bool]"
/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/torch/include/ATen/core/ivalue.h(711): here

/usr/include/c++/11/type_traits(1406): error: type name is not allowed
detected during:
instantiation of class "std::is_same<_Tp, _Up> [with _Tp=c10::IValue, _Up=c10::IValue]"
/usr/include/c++/11/bits/stl_vector.h(401): here
instantiation of class "std::vector<_Tp, _Alloc> [with _Tp=c10::IValue, _Alloc=std::allocatorc10::IValue]"
/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/torch/include/ATen/core/List.h(31): here

/usr/include/c++/11/type_traits(1406): error: type name is not allowed
detected during:
instantiation of class "std::is_same<_Tp, _Up> [with _Tp=c10::IValue, _Up=c10::IValue]"
/usr/include/c++/11/bits/stl_vector.h(401): here
instantiation of class "std::vector<_Tp, _Alloc> [with _Tp=c10::IValue, _Alloc=std::allocatorc10::IValue]"
/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/torch/include/ATen/core/List.h(31): here

/usr/include/c++/11/type_traits(1406): error: type name is not allowed
detected during:
instantiation of class "std::is_same<_Tp, _Up> [with _Tp=std::string, _Up=c10::nullopt_t]"
/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/torch/include/c10/util/Optional.h(455): here
instantiation of class "c10::optional [with T=std::string]"
/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/torch/include/ATen/core/jit_type_base.h(107): here

/usr/include/c++/11/type_traits(1406): error: type name is not allowed
detected during:
instantiation of class "std::is_same<_Tp, _Up> [with _Tp=std::string, _Up=c10::nullopt_t]"
/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/torch/include/c10/util/Optional.h(455): here
instantiation of class "c10::optional [with T=std::string]"
/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/torch/include/ATen/core/jit_type_base.h(107): here

/usr/include/c++/11/type_traits(1406): error: type name is not allowed
detected during:
instantiation of class "std::is_same<_Tp, _Up> [with _Tp=std::string, _Up=c10::in_place_t]"
/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/torch/include/c10/util/Optional.h(458): here
instantiation of class "c10::optional [with T=std::string]"
/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/torch/include/ATen/core/jit_type_base.h(107): here

/usr/include/c++/11/type_traits(1406): error: type name is not allowed
detected during:
instantiation of class "std::is_same<_Tp, _Up> [with _Tp=std::string, _Up=c10::in_place_t]"
/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/torch/include/c10/util/Optional.h(458): here
instantiation of class "c10::optional [with T=std::string]"
/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/torch/include/ATen/core/jit_type_base.h(107): here

/usr/include/c++/11/type_traits(1406): error: type name is not allowed
detected during instantiation of class "std::is_same<_Tp, _Up> [with _Tp=c10::optionalstd::string, _Up=c10::in_place_t]"
/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/torch/include/ATen/core/jit_type_base.h(107): here

/usr/include/c++/11/type_traits(1406): error: type name is not allowed
detected during instantiation of class "std::is_same<_Tp, _Up> [with _Tp=c10::optionalstd::string, _Up=c10::in_place_t]"
/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/torch/include/ATen/core/jit_type_base.h(107): here

/usr/include/c++/11/type_traits(1406): error: type name is not allowed
detected during instantiation of class "std::is_same<_Tp, _Up> [with _Tp=c10::optionalstd::string, _Up=c10::optionalstd::string]"
/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/torch/include/ATen/core/jit_type_base.h(107): here

/usr/include/c++/11/type_traits(1406): error: type name is not allowed
detected during instantiation of class "std::is_same<_Tp, _Up> [with _Tp=c10::optionalstd::string, _Up=c10::optionalstd::string]"
/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/torch/include/ATen/core/jit_type_base.h(107): here

/usr/include/c++/11/type_traits(1406): error: type name is not allowed
detected during instantiation of class "std::is_same<_Tp, _Up> [with _Tp=c10::TypePrinter, _Up=std::function<c10::optionalstd::string (const c10::ConstTypePtr &)>]"
/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/torch/include/ATen/core/jit_type_base.h(111): here

/usr/include/c++/11/type_traits(1406): error: type name is not allowed
detected during instantiation of class "std::is_same<_Tp, _Up> [with _Tp=c10::TypePrinter, _Up=std::function<c10::optionalstd::string (const c10::ConstTypePtr &)>]"
/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/torch/include/ATen/core/jit_type_base.h(111): here

/usr/include/c++/11/type_traits(1406): error: type name is not allowed
detected during instantiation of class "std::is_same<_Tp, _Up> [with _Tp=std::nullptr_t, _Up=std::function<c10::optionalstd::string (const c10::ConstTypePtr &)>]"
/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/torch/include/ATen/core/jit_type_base.h(116): here

/usr/include/c++/11/type_traits(1406): error: type name is not allowed
detected during instantiation of class "std::is_same<_Tp, _Up> [with _Tp=std::nullptr_t, _Up=std::function<c10::optionalstd::string (const c10::ConstTypePtr &)>]"
/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/torch/include/ATen/core/jit_type_base.h(116): here

/usr/include/c++/11/type_traits(1406): error: type name is not allowed
detected during:
instantiation of class "std::is_same<_Tp, _Up> [with _Tp=c10::TypePtr, _Up=c10::TypePtr]"
/usr/include/c++/11/bits/stl_vector.h(401): here
instantiation of class "std::vector<_Tp, _Alloc> [with _Tp=c10::TypePtr, _Alloc=std::allocatorc10::TypePtr]"
/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/torch/include/ATen/core/jit_type_base.h(204): here

/usr/include/c++/11/type_traits(1406): error: type name is not allowed
detected during:
instantiation of class "std::is_same<_Tp, _Up> [with _Tp=c10::TypePtr, _Up=c10::TypePtr]"
/usr/include/c++/11/bits/stl_vector.h(401): here
instantiation of class "std::vector<_Tp, _Alloc> [with _Tp=c10::TypePtr, _Alloc=std::allocatorc10::TypePtr]"
/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/torch/include/ATen/core/jit_type_base.h(204): here

/usr/include/c++/11/type_traits(1406): error: type name is not allowed
detected during:
instantiation of class "std::is_same<_Tp, _Up> [with _Tp=std::reference_wrapper, _Up=c10::trivially_copyable_optimization_optional_base<std::reference_wrapper>]"
(946): here
instantiation of class "std::__is_constructible_impl<_Tp, _Args...> [with _Tp=c10::trivially_copyable_optimization_optional_base<std::reference_wrapper>, _Args=<const c10::trivially_copyable_optimization_optional_base<std::reference_wrapper> &>]"
(978): here
instantiation of class "std::__is_copy_constructible_impl<_Tp, true> [with _Tp=c10::trivially_copyable_optimization_optional_base<std::reference_wrapper>]"
(985): here
instantiation of class "std::is_copy_constructible<_Tp> [with _Tp=c10::trivially_copyable_optimization_optional_base<std::reference_wrapper>]"
/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/torch/include/c10/util/Optional.h(408): here
instantiation of type "c10::OptionalBase<std::reference_wrapper>"
/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/torch/include/c10/util/Optional.h(427): here
instantiation of class "c10::optional [with T=std::reference_wrapper]"
/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/torch/include/ATen/core/List_inl.h(180): here

/usr/include/c++/11/type_traits(1406): error: type name is not allowed
detected during:
instantiation of class "std::is_same<_Tp, _Up> [with _Tp=std::reference_wrapper, _Up=c10::trivially_copyable_optimization_optional_base<std::reference_wrapper>]"
(946): here
instantiation of class "std::__is_constructible_impl<_Tp, _Args...> [with _Tp=c10::trivially_copyable_optimization_optional_base<std::reference_wrapper>, _Args=<const c10::trivially_copyable_optimization_optional_base<std::reference_wrapper> &>]"
(978): here
instantiation of class "std::__is_copy_constructible_impl<_Tp, true> [with _Tp=c10::trivially_copyable_optimization_optional_base<std::reference_wrapper>]"
(985): here
instantiation of class "std::is_copy_constructible<_Tp> [with _Tp=c10::trivially_copyable_optimization_optional_base<std::reference_wrapper>]"
/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/torch/include/c10/util/Optional.h(408): here
instantiation of type "c10::OptionalBase<std::reference_wrapper>"
/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/torch/include/c10/util/Optional.h(427): here
instantiation of class "c10::optional [with T=std::reference_wrapper]"
/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/torch/include/ATen/core/List_inl.h(180): here

/usr/include/c++/11/type_traits(1406): error: type name is not allowed
detected during:
instantiation of class "std::is_same<_Tp, _Up> [with _Tp=std::reference_wrapper, _Up=c10::nullopt_t]"
/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/torch/include/c10/util/Optional.h(455): here
instantiation of class "c10::optional [with T=std::reference_wrapper]"
/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/torch/include/ATen/core/List_inl.h(180): here

/usr/include/c++/11/type_traits(1406): error: type name is not allowed
detected during:
instantiation of class "std::is_same<_Tp, _Up> [with _Tp=std::reference_wrapper, _Up=c10::nullopt_t]"
/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/torch/include/c10/util/Optional.h(455): here
instantiation of class "c10::optional [with T=std::reference_wrapper]"
/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/torch/include/ATen/core/List_inl.h(180): here

/usr/include/c++/11/type_traits(1406): error: type name is not allowed
detected during:
instantiation of class "std::is_same<_Tp, _Up> [with _Tp=std::reference_wrapper, _Up=c10::in_place_t]"
/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/torch/include/c10/util/Optional.h(458): here
instantiation of class "c10::optional [with T=std::reference_wrapper]"
/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/torch/include/ATen/core/List_inl.h(180): here

/usr/include/c++/11/type_traits(1406): error: type name is not allowed
detected during:
instantiation of class "std::is_same<_Tp, _Up> [with _Tp=std::reference_wrapper, _Up=c10::in_place_t]"
/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/torch/include/c10/util/Optional.h(458): here
instantiation of class "c10::optional [with T=std::reference_wrapper]"
/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/torch/include/ATen/core/List_inl.h(180): here

/usr/include/c++/11/type_traits(1406): error: type name is not allowed
detected during:
instantiation of class "std::is_same<_Tp, _Up> [with _Tp=std::reference_wrapper, _Up=c10::optional<std::reference_wrapper>]"
(946): here
instantiation of class "std::__is_constructible_impl<_Tp, _Args...> [with _Tp=std::reference_wrapper, _Args=<c10::optional<std::reference_wrapper> &&>]"
(953): here
instantiation of class "std::is_constructible<_Tp, _Args...> [with _Tp=std::reference_wrapper, _Args=<c10::optional<std::reference_wrapper> &&>]"
/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/torch/include/ATen/core/List_inl.h(181): here

/usr/include/c++/11/type_traits(1406): error: type name is not allowed
detected during:
instantiation of class "std::is_same<_Tp, _Up> [with _Tp=std::reference_wrapper, _Up=c10::optional<std::reference_wrapper>]"
(946): here
instantiation of class "std::__is_constructible_impl<_Tp, _Args...> [with _Tp=std::reference_wrapper, _Args=<c10::optional<std::reference_wrapper> &&>]"
(953): here
instantiation of class "std::is_constructible<_Tp, _Args...> [with _Tp=std::reference_wrapper, _Args=<c10::optional<std::reference_wrapper> &&>]"
/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/torch/include/ATen/core/List_inl.h(181): here

Error limit reached.
100 errors detected in the compilation of "/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/nncf/torch/extensions/src/quantization/cuda/functions_cuda_impl.cu".
Compilation terminated.
[2/3] c++ -MMD -MF functions_cuda.o.d -DTORCH_EXTENSION_NAME=quantized_functions_cuda -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE="_gcc" -DPYBIND11_STDLIB="_libstdcpp" -DPYBIND11_BUILD_ABI="_cxxabi1011" -I/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/nncf/torch/extensions/include -isystem /home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/torch/include -isystem /home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/torch/include/torch/csrc/api/include -isystem /home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/torch/include/TH -isystem /home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/torch/include/THC -isystem /home/chenxc/anaconda3/envs/Grounding/include/python3.8 -D_GLIBCXX_USE_CXX11_ABI=0 -fPIC -std=c++14 -c /home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/nncf/torch/extensions/src/quantization/cuda/functions_cuda.cpp -o functions_cuda.o
ninja: build stopped: subcommand failed.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "run.py", line 87, in
main(args)
File "run.py", line 76, in main
run_train(args)
File "/home/chenxc/work/HvTrainTools/engine/init.py", line 24, in run_train
itrainer = trainer.init_trainer(args,params_json,device,opt)
File "/home/chenxc/work/HvTrainTools/engine/train/basic_trainer.py", line 35, in init_trainer
return Trainer(args,params_json,device,opt)
File "/home/chenxc/work/HvTrainTools/engine/train/basic_trainer.py", line 126, in init
self.quant_model()
File "/home/chenxc/work/HvTrainTools/engine/train/basic_trainer.py", line 227, in quant_model
quantized_model = nncf.quantize(self.model, calibration_dataset)#,ignored_scope=nncf.IgnoredScope(names=['FeatextractionModel/UniversalCircleMVSoftmax[head]/flatten_0']))
File "/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/nncf/quantization/quantize_model.py", line 123, in quantize
return quantize_impl(
File "/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/nncf/torch/quantization/quantize_model.py", line 253, in quantize_impl
compression_ctrl, compressed_model = create_compressed_model(
File "/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/nncf/telemetry/decorator.py", line 71, in wrapped
retval = fn(*args, **kwargs)
File "/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/nncf/torch/model_creation.py", line 117, in create_compressed_model
compressed_model = builder.apply_to(nncf_network)
File "/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/nncf/torch/compression_method_api.py", line 127, in apply_to
self.initialize(transformed_model)
File "/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/nncf/torch/quantization/algo.py", line 1289, in initialize
bn_adaptation.run(model)
File "/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/nncf/common/initialization/batchnorm_adaptation.py", line 100, in run
impl.run(model)
File "/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/nncf/torch/batchnorm_adaptation.py", line 28, in run
bn_adaptation_runner.run(self._data_loader, self._num_bn_adaptation_steps)
File "/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/nncf/torch/initialization.py", line 149, in run
self._run_model_inference(data_loader, num_init_steps, device)
File "/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/nncf/torch/initialization.py", line 218, in _run_model_inference
self._infer_batch(args_kwargs_tuple, device)
File "/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/nncf/torch/initialization.py", line 137, in _infer_batch
self.model(*args, **kwargs)
File "/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/nncf/torch/nncf_network.py", line 886, in call
return ORIGINAL_CALL(self, *args, **kwargs)
File "/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1051, in _call_impl
return forward_call(*input, **kwargs)
File "/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/nncf/torch/nncf_network.py", line 901, in forward
args, kwargs = self.nncf._wrap_inputs_fn(args, kwargs)
File "/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/nncf/torch/quantization/quantize_model.py", line 226, in wrap_inputs
return wrap_nncf_model_inputs_with_objwalk(args, kwargs)
File "/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/nncf/torch/dynamic_graph/io_handling.py", line 35, in wrap_nncf_model_inputs_with_objwalk
model_args = objwalk(model_args, is_tensor, nncf_model_input)
File "/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/nncf/torch/nested_objects_traversal.py", line 161, in objwalk
obj[idx] = apply_fn(obj[idx])
File "/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/nncf/torch/dynamic_graph/wrappers.py", line 94, in wrapped
result = _execute_op(op_address, operator_info, operator, ctx, *args, **kwargs)
File "/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/nncf/torch/dynamic_graph/wrappers.py", line 192, in _execute_op
result = ctx.execute_post_hooks(op_address, result)
File "/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/nncf/torch/dynamic_graph/context.py", line 290, in execute_post_hooks
outputs = hook(outputs)
File "/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/nncf/torch/quantization/algo.py", line 884, in call
return storage[self.quantizer_storage_key](*args, **kwargs)
File "/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/nncf/torch/dynamic_graph/wrappers.py", line 151, in wrapped
retval = module_call(self, *args, **kwargs)
File "/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1051, in _call_impl
return forward_call(*input, **kwargs)
File "/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/nncf/torch/quantization/layers.py", line 398, in forward
return self.quantize(x, execute_traced_op_as_identity=False)
File "/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/nncf/torch/quantization/layers.py", line 711, in quantize
return symmetric_quantize(
File "/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/nncf/torch/dynamic_graph/wrappers.py", line 94, in wrapped
result = execute_op(op_address, operator_info, operator, ctx, *args, **kwargs)
File "/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/nncf/torch/dynamic_graph/wrappers.py", line 175, in execute_op
result = operator(*args, **kwargs)
File "/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/nncf/torch/quantization/quantize_functions.py", line 197, in symmetric_quantize
return QuantizeSymmetric.apply(input, scale_safe, level_low, level_high, levels)
File "/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/nncf/torch/quantization/quantize_functions.py", line 41, in forward
output = QuantizedFunctionsCUDA.get("Quantize_forward")(input, input_low, input_range, levels)
File "/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/nncf/torch/extensions/init.py", line 74, in get
self._loaded_namespace = self._loader.load()
File "/home/chenxc/anaconda3/envs/Grounding/lib/python3.8/site-packages/nncf/torch/quantization/extensions.py", line 92, in load
raise RuntimeError(
RuntimeError: CUDA is available for PyTorch, but NNCF could not compile GPU quantization extensions. Make sure that you have installed CUDA development tools (see https://docs.nvidia.com/cuda/cuda-installation-guide-linux/index.html for guidance) and that 'nvcc' is available on your system's PATH variable.

[vshampor]

A short google search for error: identifier "__is_same" is undefined yields several reports of incompatible GCC and CUDA combinations. @huangqiu15444 I suggest you do the same, review the reports and check whether your GCC and CUDA versions are known to be compatible.

[MaximProshin]

@huangqiu15444 , do you have any update here? Did you check what @vshampor recommended above about the compatibility?