Let’s say you want to create a document on your Office 365 public website. You go to Site Actions, click on View All Site Content, click on Create, and create a new document library.
At the end of that, you get an error: 
Checking your website, however, you see that the document library was created nevertheless. 
But once you open it, you realise that you can’t access its settings, the Library tab is missing from the Ribbon! 
What to do now? I show you how you can still go to the library’s settings page, and as a last step delete the library.
First, notice that small little dropdown next to All Documents in the previous screenshot? Click on it, and select Modify this View 
You’re now on the edit page for the view. But behold, what’s that in the navigation now? The Document Library Settings! Click on it! 
No surprise, you can now see your library’s settings. Click on Delete this document library to, well, delete it. 
This guide has helped me remove a few of the issued libraries, However there is one of the libraries (first I made) that doesn’t have the Delete functionality in the Permissions and Management column.
also when browsing to the library in the admin toll for the site, the library only has the browse tab in the ribbon nothing else. first it had pages tab in it but it has since then disapeared.
This might be because the library was created as a webpages library, and for some reason Sharpoint does not allow removal of this type of library.
Hi Stefan,
thanks for sharing that information. I think because creating document libraries on the public website is not a “support/desired” behaviour by Microsoft, these issues can happen. I’ve come to live with the fact that the things I do with my public website (activating Publishing feature, adding lists & libraries) are not supported, that they may not work perfectly, and that a future update may actually ‘destroy’ them