Reason: The ec2-user doesn’t have permission on that directory.
Solution:
sudo su - to become root. Then you’ll be able to go into the directory and do as you see fit.
sudo chmod 755 /home/ec2-user which will make that user directory world readable.
If the ec2-user is a real user (i.e. has an entry in /etc/passwd) then you can sudo su - ec2-user to become that operating system user.
The relative merits of the answers depend on the security that you want in your system. If you just need to look then the first or third answer are likely the best as they are not a persistent change. But if you really want to use that directory by the ec2-user then you may want to use the second answer.
Which Spark kernel to use on AWS SageMaker?
Use python3 and import pysaprk
If you want to use spark magic
You should also start Livy which is API service used by magicspark to talk to your Spark cluster.
How?
Download Livy from Apache Livy and unzip it
Check SPARK_HOME environment is set, if not, set to your Spark
Run Livy server by <livy_home>/bin/livy-server in the command line