Test loading of package on unsupported platforms

[christiangnrd]

Close #508

[codecov]

Codecov Report

Attention: Patch coverage is 50.00000% with 1 line in your changes missing coverage. Please review.

Project coverage is 77.29%. Comparing base (52d7056) to head (a25e564).
Report is 392 commits behind head on main.

Files with missing lines	Patch %	Lines
src/state.jl	0.00%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #509      +/-   ##
==========================================
+ Coverage   71.04%   77.29%   +6.25%     
==========================================
  Files          36       57      +21     
  Lines        1143     2722    +1579     
==========================================
+ Hits          812     2104    +1292     
- Misses        331      618     +287     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[github-actions]

Metal Benchmarks

Benchmark suite	Current: a25e564	Previous: cf21f9d	Ratio
private array/construct	27458.333333333332 ns	27517.85714285714 ns	1.00
private array/broadcast	463750 ns	457584 ns	1.01
private array/random/randn/Float32	785979 ns	821104.5 ns	0.96
private array/random/randn!/Float32	638709 ns	616500 ns	1.04
private array/random/rand!/Int64	553041 ns	548167 ns	1.01
private array/random/rand!/Float32	588083 ns	582125 ns	1.01
private array/random/rand/Int64	737062.5 ns	764500 ns	0.96
private array/random/rand/Float32	597895.5 ns	598542 ns	1.00
private array/copyto!/gpu_to_gpu	656416 ns	646125 ns	1.02
private array/copyto!/cpu_to_gpu	817291 ns	810041.5 ns	1.01
private array/copyto!/gpu_to_cpu	633625 ns	644937.5 ns	0.98
private array/accumulate/1d	1318250 ns	1320000 ns	1.00
private array/accumulate/2d	1377708.5 ns	1378958 ns	1.00
private array/iteration/findall/int	2106333 ns	2020834 ns	1.04
private array/iteration/findall/bool	1821208.5 ns	1779625 ns	1.02
private array/iteration/findfirst/int	1689583 ns	1696916 ns	1.00
private array/iteration/findfirst/bool	1660542 ns	1650188 ns	1.01
private array/iteration/scalar	3637750 ns	3549500 ns	1.02
private array/iteration/logical	3163792 ns	3174729.5 ns	1.00
private array/iteration/findmin/1d	1755708 ns	1747792 ns	1.00
private array/iteration/findmin/2d	1336354 ns	1339417 ns	1.00
private array/reductions/reduce/1d	1045500 ns	1031312.5 ns	1.01
private array/reductions/reduce/2d	663333 ns	656042 ns	1.01
private array/reductions/mapreduce/1d	1036459 ns	1027125 ns	1.01
private array/reductions/mapreduce/2d	658167 ns	666791 ns	0.99
private array/permutedims/4d	2553563 ns	2554792 ns	1.00
private array/permutedims/2d	1016917 ns	1026291.5 ns	0.99
private array/permutedims/3d	1583583 ns	1591209 ns	1.00
private array/copy	576979.5 ns	595583 ns	0.97
latency/precompile	5845227958.5 ns	5856015937.5 ns	1.00
latency/ttfp	6701292250 ns	6607073479.5 ns	1.01
latency/import	1168569834 ns	1171437917 ns	1.00
integration/metaldevrt	712708 ns	699417 ns	1.02
integration/byval/slices=1	1632584 ns	1594083.5 ns	1.02
integration/byval/slices=3	9686020.5 ns	11850083 ns	0.82
integration/byval/reference	1611833.5 ns	1622417 ns	0.99
integration/byval/slices=2	2559583 ns	2525791 ns	1.01
kernel/indexing	467521 ns	466979.5 ns	1.00
kernel/indexing_checked	480083 ns	479562.5 ns	1.00
kernel/launch	7959 ns	7958 ns	1.00
metal/synchronization/stream	15250 ns	14708 ns	1.04
metal/synchronization/context	15541 ns	14834 ns	1.05
shared array/construct	27118 ns	25312.5 ns	1.07
shared array/broadcast	477542 ns	466145.5 ns	1.02
shared array/random/randn/Float32	851167 ns	784895.5 ns	1.08
shared array/random/randn!/Float32	627333 ns	648729 ns	0.97
shared array/random/rand!/Int64	555542 ns	543291 ns	1.02
shared array/random/rand!/Float32	604042 ns	579958.5 ns	1.04
shared array/random/rand/Int64	772958 ns	764687.5 ns	1.01
shared array/random/rand/Float32	616916 ns	636958 ns	0.97
shared array/copyto!/gpu_to_gpu	87042 ns	87791 ns	0.99
shared array/copyto!/cpu_to_gpu	87875 ns	94417 ns	0.93
shared array/copyto!/gpu_to_cpu	78250 ns	83333 ns	0.94
shared array/accumulate/1d	1342645.5 ns	1327875 ns	1.01
shared array/accumulate/2d	1377417 ns	1378167 ns	1.00
shared array/iteration/findall/int	1763708 ns	1773979 ns	0.99
shared array/iteration/findall/bool	1585542 ns	1575646 ns	1.01
shared array/iteration/findfirst/int	1389958 ns	1391812 ns	1.00
shared array/iteration/findfirst/bool	1356042 ns	1352062.5 ns	1.00
shared array/iteration/scalar	159666 ns	155709 ns	1.03
shared array/iteration/logical	2963125 ns	2959083.5 ns	1.00
shared array/iteration/findmin/1d	1460250 ns	1459687.5 ns	1.00
shared array/iteration/findmin/2d	1358416 ns	1353292 ns	1.00
shared array/reductions/reduce/1d	736125 ns	727896 ns	1.01
shared array/reductions/reduce/2d	666687.5 ns	660583 ns	1.01
shared array/reductions/mapreduce/1d	744125 ns	730833 ns	1.02
shared array/reductions/mapreduce/2d	670834 ns	662708.5 ns	1.01
shared array/permutedims/4d	2555542 ns	2539083.5 ns	1.01
shared array/permutedims/2d	1018958 ns	1019208 ns	1.00
shared array/permutedims/3d	1589000 ns	1600708 ns	0.99
shared array/copy	246167 ns	238729.5 ns	1.03

This comment was automatically generated by workflow using github-action-benchmark.

[maleadt]

Interesting approach. This should make it possible to remove the PkgEval blacklist, so that package loadability is also tested there. I wonder if we need to do something similar with CUDA.jl and the other back-ends.

[maleadt]

On macos-13, this doesn't really just test Loading though but runs all tests. So maybe remove that platform?

EDIT: ah, I guess this does bail out because of being a virtualized platform. I wonder if the error thrown by the test suite shouldn't be a bit more verbose, though (i.e., mentioning the specific reason).

[christiangnrd]

I fixed up the version check and moved it from setup.jl to runtests.jl so that it only runs once. If there was a specific reason that the check was in setup.jl I can move it back.

[christiangnrd]

@maleadt I made the support check message more verbose for paravirtual devices.

I also added a helper function for safely checking the lower bound on all OSes as this'll become useful as we add more API wrappers for macos 14 and 15 features (like the MTLArchitecture wrapper I also snuck into this PR). Not duplicating all of MTLDevice to add the architecture::MTLArchitecture property that only exists since macOS 14 is what inspired JuliaInterop/ObjectiveC.jl#46

[christiangnrd]

Failure is the riscv bug

[christiangnrd]

Accidentally deleted branch, reopening.