2023.4.28.0

Plugins
IPluginContentProvider: added DownloadSingleObject function; added tokens to GetMedia and Download functions; removed GetSpecialData function
Add IDownloadableMedia interface
Removed 'Channel' option from all functions and enums
ISiteSettings: added GetSingleMediaInstance function
ExchangeOptions: removed 'IsChannel'
UserMediaTypes: added Audio and AudioPre enums
IUserMedia, PluginUserMedia: changed ContentType and DownloadState from integers to their enums

SCrawler
Add YouTube standalone downloader
Add gallery-dl & yt-dlp support
Remove 'UserInfo' requirement from 'ProfilesSaved'
Update 'SiteSettingsBase' to use domains and Netscape cookies
UserDataBase: remove channels; remove old 'Merge' const; standardize SavedPosts file naming; move 'ValidateMD5' function from Twitter to UserDataBase to use it in other UserData classes; add 'DownloadSingleObject' environment for single posts; add validating file extension for m3u8 during download; add reindex of video file during download

Rewritten DomainsContainer
Create a universal settings form and PSettingsArttribute
Gfycat, Imgur: turn these classes into IUserData to download a single object

All plugins: update 'GetInstance' function for saved posts; update domains where implemented; remove 'OptionForm' where it exists; update options where they exist; update unix date providers; reconfigure channels where they exist

LPSG: fix attachments; update converters and regex
Add sites: ThisVid, Mastodon, Pinterest, YouTube, YouTube music
Reddit: standardize container parsing for all data types; new channel environment; fix 'ReparseMissing' function; redirect data downloading to the base download function, saved crossposts support
Twitter: fixed gif path bug; fixed downloading saved posts
PornHub: hide unnecessary errors; photo galleries bug
RedGifs: add 'UserAgent' option

Added icons to download progress

Rename some objects
Completely redesigned standalone downloader form and rewritten its environment
WebClient2: update to use tokens

Labels: update label form (save labels to file only when OK button is clicked); change removing labels.txt from recycle bin to permanent; disable storing label 'NoParsedUser'

UserCreatorForm: remove the 'Channel' checkbox and related functions; ability to extract the user's URL from the buffer and apply parameters if found
Remove temporary 'EncryptCookies' module

MainFrame: added simplified way to create new users (Ctrl+Insert to create a new user with default parameters from clipboard URL); removed SCrawler command line argument "-v" (remove the ability to run SCrawler as video downloader)
PropertyValueHost: update for option forms compatibility
SettingsHost: removed 'GetSpecialData' fork; added 'GetSingleMediaInstance' fork
UserDataHost: update functions with tokens; update events; add 'DownloadSingleObject' function
Settings: add the ability to get environment from 4 destinations; add the ability to set the program environment manually; add CMDEncoding; add cache; remove the old function 'RemoveUnusedPlugins'; add 'STDownloader' properties; add YT compatibility; add new notification options; add deleting user settings file when 'SettingsCLS.Dispose()' if where are no users in SCrawler
UserFinder: remove old 'Merge' const; remove channel option
UserInfo: remove channel option

Author: AAndyProgram

.gitattributes

@@ -1,2 +1,5 @@
 # Auto detect text files and perform LF normalization
 * text=auto
+
+# Declare files that will always have CRLF line endings on checkout.
+*.md text eol=crlf

.github/FUNDING.yml

@@ -3,11 +3,11 @@
 github: # Replace with up to 4 GitHub Sponsors-enabled usernames e.g., [user1, user2]
 patreon: # Replace with a single Patreon username
 open_collective: # Replace with a single Open Collective username
-ko_fi: andyprogram
+ko_fi: #andyprogram
 tidelift: # Replace with a single Tidelift platform-name/package-name e.g., npm/babel
 community_bridge: # Replace with a single Community Bridge project-name e.g., cloud-foundry
 liberapay: # Replace with a single Liberapay username
 issuehunt: # Replace with a single IssueHunt username
 otechie: # Replace with a single Otechie username
 lfx_crowdfunding: # Replace with a single LFX Crowdfunding project-name e.g., cloud-foundry
-custom: ['https://blockchair.com/bitcoin/address/BC1Q0NH839FT5TA44DD7L7RLR97XDQAG9V8D6N7XET']
+custom: ['https://github.com/AAndyProgram/SCrawler/blob/main/HowToSupport.md']

.github/ISSUE_TEMPLATE/bug_report.md

@@ -13,11 +13,16 @@ A clear and concise description of what the bug is.
 **To Reproduce**
 Steps to reproduce the behavior:
 1. **Profile URL**: 
-2. Do something
-3. See error
+2. **Post URL**: 
+3. Do something
+4. See error
 
-**Log data**
-If the program log contains any data.
+<details>
+<summary>Log data</summary>
+<pre>
+If the program log contains any data, replace this line with the log data. If the program log does not contain any data, write here about.
+</pre>
+</details>
 
 **Expected behavior**
 A clear and concise description of what you expected to happen.
@@ -26,10 +31,13 @@ A clear and concise description of what you expected to happen.
 If applicable, add screenshots to help explain your problem.
 
 **Release information (please complete the following information):**
- - OS [e.g. Windows 10, Windows 11]
- - Architecture [e.g. x86, x64]
- - Version [e.g. 2.0.0.0]
- - NET.Framework version
+ - OS: [e.g. Windows 10, Windows 11]
+ - Architecture: [e.g. x86, x64]
+ - Version: [e.g. 2023.3.5.0]
+ - NET.Framework version: 
+ - ffmpeg version (command `ffmpeg -version`):
+ - yt-dlp version (command `yt-dlp --version`):
+ - gallery-dl version (command `gallery-dl --version`):
 
 **Additional context**
 Add any other context about the problem here.

.gitignore

@@ -9,8 +9,9 @@
 *.user
 *.userosscache
 *.sln.docstates
-ffmpeg.exe
+.obsidian/
 ToDo.txt
+ToDo.md
 
 # User-specific files (MonoDevelop/Xamarin Studio)
 *.userprefs
@@ -33,10 +34,7 @@ bld/
 [Oo]bj/
 [Ll]og/
 [Ll]ogs/
-ffmpeg/
-cURL/
 Info/
-Hidden/
 
 # Visual Studio 2015/2017 cache/options directory
 .vs/

CONTRIBUTING.md

@@ -8,20 +8,21 @@ I welcome requests! Follow these steps to contribute:
 1. If you have a code change suggestion, you can post a replacement code block. I also accept pull requests.
 
 # How to build from source
-1. Delete the ```PersonalUtilities``` project from the solution.
-1. Delete the ```PersonalUtilities.Notifications``` project from the solution.
-1. Delete the ```cURL``` folder from the solution.
-1. Delete the ```ffmpeg.exe``` from the solution.
+1. Delete the `PersonalUtilities` project from the solution.
+1. Delete the `PersonalUtilities.Notifications` project from the solution.
 1. The following libraries must be added to project references with the '**Copy to output folder**' option:
-    - ```PersonalUtilities.dll```
-    - ```PersonalUtilities.Notifications.dll```
-    - ```Microsoft.Toolkit.Uwp.Notifications.dll```
-    - ```System.ValueTuple.dll```
-1. Import ```PersonalUtilities.Functions``` for the whole project.
+    - `PersonalUtilities.dll`
+    - `PersonalUtilities.Notifications.dll`
+    - `Microsoft.Toolkit.Uwp.Notifications.dll`
+    - `System.ValueTuple.dll`
+1. Import `PersonalUtilities.Functions` for the whole project.
 
 **Always use the correct libraries. You must download libraries from the same release date as the code commit date.**
 
 # How to request a new site
+
+**I'm currently not accepting requests to develop new sites.**
+
 1. Check [issues](https://github.com/AAndyProgram/SCrawler/issues) (open and [closed](https://github.com/AAndyProgram/SCrawler/issues?q=is%3Aissue+is%3Aclosed)) and [discussions](https://github.com/AAndyProgram/SCrawler/discussions) to find your issue. Perhaps I have already answered your request.
 1. If you don't find anything, create a new issue with your request. I usually reply as soon as possible (within the next few hours).
 

Changelog.md

@@ -1,3 +1,47 @@
+# 2023.4.28.0
+
+*2023-04-28*
+
+- Added
+  - **YouTube**
+  - **YouTube Music**
+  - **Mastodon**
+  - **Pinterest**
+  - **ThisVid**
+  - **YouTube downloader (standalone app)**
+  - Redesigned standalone downloader and update environment
+  - Added icons to download progress
+  - Added icons to saved posts downloader
+  - **Cookies**: new ways to add cookies. You can now export cookies using the browser extension and then import them into SCrawler!
+  - User creation: ability to extract the user's URL from the buffer and apply parameters if found
+  - User creation: simplified way to create new users (`Ctrl+Insert` to create a new user with default parameters from clipboard URL)
+  - Ability to customize the placement of ffmpeg (and other) files
+  - Ability to customize the command line encoding
+  - New notification options for standalone downloader
+  - Reddit: now it can download saved crossposts
+  - RedGifs: added `UserAgent` option
+  - Other improvements
+- Removed
+  - User creation: remove the 'Channel' checkbox because it confuses people
+  - Removed an ability to open SCrawler with `-v` argument
+  - All ways to create users except URL. You can only properly create a user using the user's URL.
+- Plugins
+  - Added `IDownloadableMedia` interface
+  - Removed `Channel` option from all functions and enums
+  - ISiteSettings: added `GetSingleMediaInstance` function
+  - IPluginContentProvider: added `DownloadSingleObject` function
+  - IPluginContentProvider: added tokens to `GetMedia` and `Download` functions
+  - IPluginContentProvider: removed `GetSpecialData` function
+  - UserMediaTypes: added `Audio` and `AudioPre` enums
+- Fixed
+  - LPSG: attachments not downloading (Issue #114)
+  - Twitter: saved posts not downloading (Issue #119)
+  - XVIDEOS: saved posts not downloading
+  - Deleting labels file
+  - PornHub: hide unnecessary errors (Issue #116)
+  - PornHub: photo galleries bug (Issue #115)
+  - Minor bugs
+
 # 2023.3.5.0
 
 *2023-03-05*

FAQ.md

@@ -14,11 +14,7 @@ Any other questions I will keep in this file.
 
 A: https://github.com/AAndyProgram/SCrawler/wiki/Settings#how-to-set-up-cookies
 
-----
-
-#### Q: **I can't copy cookies.**
-
-A: Use the mouse. Don't use ```Ctrl+A```!
+<!---**ATTENTION! If you need to use cookies but cannot import them, I highly recommend that you don't use SCrawler and use another program. Don't create issues, discussions, or write to me on Discord. Any issue or discussion about cookies will be deleted immediately without a response. Any user who asks me about cookies on Discord will be banned.**--->
 
 ----
 
@@ -30,21 +26,22 @@ A: This is a GUI program.
 
 #### Q: **Will CLI be added in the future?**
 
-A: I do not think so.
+A: NO.
 
 ----
 
 #### Q: **I want to add "...." site. How to request.**
 
-A: How to request a new site you can read [here](CONTRIBUTING.md#how-to-request-a-new-site)
+<!---A: How to request a new site you can read [here](CONTRIBUTING.md#how-to-request-a-new-site)--->
+**I'm currently not accepting requests to develop new sites.**
 
 ----
 
-#### Q: **Twitter/Instagram download failed.**
+#### Q: **Site download failed.**
 
-A: Check your credentials. Both of these sites require cookies. Check your [Twitter tokens](https://github.com/AAndyProgram/SCrawler/wiki/Settings#how-to-find-twitter-tokens) and [Instagram settings](https://github.com/AAndyProgram/SCrawler/wiki/Settings#instagram). If all settings are set, but nothing works, go to [create a new issue](https://github.com/AAndyProgram/SCrawler/issues). Don't forget to attach the LOG.
+A: Check your credentials and **[SITES REQUIREMENTS](https://github.com/AAndyProgram/SCrawler/wiki/Settings#sites-requirements)**. If all settings are set, but nothing works, go to [create a new issue](https://github.com/AAndyProgram/SCrawler/issues). Don't forget to attach the LOG.
 
-**[SITES REQUIREMENTS](https://github.com/AAndyProgram/SCrawler/wiki/Settings#sites-requirements)**
+**ATTENTION! Issues without URLs will be closed without a response!**
 
 ----
 
@@ -80,7 +77,7 @@ A: The program stored posts IDs in users' folders. For the first time, the progr
 
 #### Q: **How to redownload all data**
 
-A: Double-click on the user you want to redownload. In the opened window open folder setting. Delete the files ending with ```_Data.xml```  and ```_Posts.txt```. Download this user again.
+A: Double-click on the user you want to redownload. In the opened window open folder setting. Delete the files ending with ```_Data.xml```  and ```_Posts.txt```. Restart SCrawler. Download this user again.
 
 ----
 
@@ -116,4 +113,4 @@ A: I can only [suggest](#q-you-lost-me-your-program-is-too-complicated) you find
 
 #### Q: **Can you add a step-by-step guide or video on how to use the program?**
 
-A: **NO**! I will not do it. If you want, you can create a video tutorial and send it to me. Then I add it. All options and what each option does described on the wiki. The wiki also contains a description of all settings and how-to configure them. For complex settings, there is a steep-by-steep guide. Read the [main](README.md) information and [GUIDE](https://github.com/AAndyProgram/SCrawler/wiki/) and you won't have any problems. I have developed a program with an intuitive interface. There is a Settings button, download buttons, a context menu that drops down when a user is clicked, and other controls. Anyone can use it.
+A: **NO! NEVER!** The guide fully covers all the functionality of SCrawler! If you don't respect my work, I don't waste my time. If you want, you can create a video tutorial and send it to me. Then I add it. All options and what each option does described on the wiki. The wiki also contains a description of all settings and how-to configure them. For complex settings, there is a steep-by-steep guide. Read the [main](README.md) information and [GUIDE](https://github.com/AAndyProgram/SCrawler/wiki/) and you won't have any problems. I have developed a program with an intuitive interface. There is a Settings button, download buttons, a context menu that drops down when a user is clicked, and other controls. Anyone can use it.

HowToSupport.md

@@ -2,11 +2,11 @@ Your support is very valuable to me. Any support is greatly appreciated. Your su
 
 You can support the program by:
   - **Bitcoin**: BC1Q0NH839FT5TA44DD7L7RLR97XDQAG9V8D6N7XET
-  - :heavy_dollar_sign: make a donation on this site: https://ko-fi.com/andyprogram
   - :repeat: make a post about my program on your profile (Reddit, Twitter, Instagram and any other social networks)
   - :speech_balloon: tell your friends about the program
   - :heart: like the program on this site: https://alternativeto.net/software/scrawler/about/
   - :heart: like the program on this site: https://www.softpedia.com/get/Internet/Download-Managers/Social-networks-crawler.shtml
   - suggest my program as an alternative ([on this site](https://alternativeto.net/software/scrawler/about/)) to any program you have used before
 
 I would be very grateful for any support! :blush:
+<!---:heavy_dollar_sign: make a donation on this site: https://ko-fi.com/andyprogram--->

ProgramScreenshots/AppYouTube.png

@@ -1,6 +1,55 @@
+# yt-dlp
+
+https://github.com/yt-dlp/yt-dlp/
+
+**Great powerful CLI tool that supports hundreds of sites.**
+
+SCrawler has advanced user management, collections, labels, groups, automatic downloads, a beautiful view, GUI, the ability to add plugins for other sites and much more. Just try it and compare.
+
+# 4K Video Downloader
+
+https://www.4kdownload.com/-plbrz/video-downloader
+
+| Option | SCrawler | 4K Stogram |
+| ---- | ---- | ---- |
+| User managament | **Advanced** | No |
+| Automatic downloads | **Yes** | No |
+| Downloading groups | **Yes** | No |
+| Labeling users | **Yes** | No |
+| Filtering | **Yes** | No |
+| Collections | **Yes** | No |
+| Specific user folders | **Yes** | No |
+| Favorite / Temporary user options | **Yes** | No |
+| Plugins support | **Yes** | No |
+| Download single video | **Unlimited** | 30 videos per day *(unlimited starts from 12 EUR)* |
+| Download videos per channel | **Unlimited** | 5 free *(unlimited starts from 12 EUR)* |
+| Download videos per playlist | **Unlimited** | 10 free *(unlimited starts from 12 EUR)* |
+| Download video subtitles | **Any for free**: single video, playlist, user/channel, album, etc| Free for single video |
+| The number of subtitles you can download for a video | **All of them** | Up to 10 |
+| Convert subtitles to additional formats | **Yes** | No |
+| Support LRC format | **Yes** | No |
+| Select audio codec for audio/video | **Yes** | No |
+| Extract and convert additional audio tracks for video | **Yes** | No |
+| Simultaneous downloads | **Unlimited** | 1 free, 3 for 12 EUR, 7 for 43 EUR|
+| Private YouTube content download | **Free** | Only in paid plans *starts from 12 EUR* |
+| **Paid** | **No** | Yes |
+| **Free options** | **The program is completely free** | Only **30** videos per day, 5 from a channel, 10 from a playlist |
+| Permitted Commercial Use | **Yes** | Starting from 43 EUR |
+| Automatic Subscriptions Update | **Free** | Paid (43 EUR) |
+| Posts and Captions Export | No | Paid (43 EUR) |
+| Advertisements free | **No ADs at all for free** | Paid (43 EUR) |
+| Operating Systems | Windows 10+ | Windows 7+, MacOS 10.13+, Ubuntu x64 |
+| Select want content type to download | **Yes** | No |
+| Instagram support | **Yes** | No |
+| Twitter support | **Yes** | No |
+| Reddit support | **Yes** | No |
+| Other sites support | **Yes** | No |
+| Still supported | Yes | Yes |
+
 # 4K Stogram
 
-https://www.4kdownload.com/products/product-stogram
+<!---https://www.4kdownload.com/products/product-stogram--->
+https://www.4kdownload.com/-ad0p9/stogram
 
 | Option | SCrawler | 4K Stogram |
 | ---- | ---- | ---- |
@@ -27,10 +76,10 @@ https://www.4kdownload.com/products/product-stogram
 | Export and import subscriptions | No | **Yes** |
 | **Paid** | **No** | Yes |
 | **Free options** | **The program is completely free** | Only **ONE** profile downloading and only **200 posts** per day |
-| Permitted Commercial Use | **Yes** | Starting from 43.56 EUR |
-| Automatic Subscriptions Update | **Free** | Paid (43.56 EUR) |
-| Posts and Captions Export | No | Paid (43.56 EUR) |
-| Advertisements free | **No ADs at all for free** | Paid (14.52) |
+| Permitted Commercial Use | **Yes** | Starting from 43 EUR |
+| Automatic Subscriptions Update | **Free** | Paid (43 EUR) |
+| Posts and Captions Export | No | Paid (43 EUR) |
+| Advertisements free | **No ADs at all for free** | Paid (18 EUR) |
 | Operating Systems | Windows 10+ | Windows 7+, MacOS 10.13+, Ubuntu x64 |
 | Select want content type to download | **Yes** | No |
 | Instagram support | Yes | Yes |
@@ -68,15 +117,14 @@ https://github.com/RipMeApp/ripme
 | **Free options** | The program is completely free | The program is completely free, but site limits are not declared |
 | Operating Systems | Windows 10+ | Windows, MacOS, Linux |
 | Select want content type to download | Yes | Yes |
-| Suported sites | 9 internal and any site using plugins | 86+ sites (declared) |
+| Suported sites | 15 internal and any site using plugins | 86+ sites (declared) |
 | Other sites support | **Yes** | No |
 | Still supported | **Yes** | **No (last release date May 4, 2021)** |
 
 # gallery-dl
 
 https://github.com/mikf/gallery-dl
 
-
-**CLI tool**! Configured with JSON files only. Users need to learn complex configuration options, JSON, commands to use that tool. Very difficult to configure.
+**CLI tool**
 
 SCrawler has advanced user management, collections, labels, groups, automatic downloads, a beautiful view, GUI, the ability to add plugins for other sites and much more. Just try it and compare.