That’s likely happening because wpsp_before_wrapper appears before the post loop is initiated.
I assume your custom field is in each post being displayed? If so, it would need to be hooked into wpsp_before_header as you found out.
What if you added the class to the post itself?:
add_filter( 'post_class', function( $classes ) {
$color = get_post_meta( get_the_ID(), '_my_color', true );
if ( $color ) {
$classes[] = $color;
}
return $classes;
} );