So, adding NO_LIBC to aws_lambda_package_target(${PROJECT_NAME}) solved my issues. I looked into packager and found that if NO_LIBC is not set, then PKG_LD remains empty, and generates bootstrap file like this:

#!/bin/bash
set -euo pipefail
export AWS_EXECUTION_ENV=lambda-cpp
exec $LAMBDA_TASK_ROOT/lib/ --library-path $LAMBDA_TASK_ROOT/lib $LAMBDA_TASK_ROOT/bin/hello ${_HANDLER}

Which of course fails. Looks pretty similar to #45 , this is on Arch Linux