When you generate a new CV the photo is not shown. The result is that you will see a grey square instead.
The external user (site user) doesn't have access to the photo.
1. Setup > Build > Create > Objects > look for the Candidate object > Page layouts > open the page layout.
2. Click in front of the page layout on Edit.
3. Look the related list Open attachments up and drag it on the page layout > Save.
Next, we need to find the name of the site user. To find this you go to Setup > in the Quick find look for sites > click on Sites > click on the site label > Public Access Settings > click on View users. In the column Full name you see the name of the site user.
Go back to the candidate record > scroll to Open attachments > open the (most recent) open attachment > click on Sharing > Add > click on Public groups and select Users (see screenshot below) > select the site user > Save. Now you have shared the photo.
If you see multiple open attachments for Contact picture you can delete the old open attachments and share the most recent one with the site user.
It is not known yet with which release the fix for this issue will be available.