summaryrefslogtreecommitdiff
path: root/apps/tagcache.c
AgeCommit message (Collapse)Author
2020-07-24[3/4] Completely remove HWCODEC supportSolomon Peachy
'swcodec' is now always set (and recording_swcodec for recording-capable units) in feature.txt so the manual and language strings don't need to all be fixed up. Change-Id: Ib2c9d5d157af8d33653e2d4b4a12881b9aa6ddb0
2019-08-20FS#9213 - Fix for audio dropout after startupIgor B. Poretsky
Original Patch by Martin Buck Updated by Igor Poretsky
2017-04-01Database: Fix destabilization if ramcache fails to load + misc.Michael Sevakis
It seems to like not having any idea any attempt was made to load it so just try to put it back like the option was never on. Integrity check other tags besides the filename. Fix a few logfs that spat out warnings and add a bit more detail to a couple. Move a buffer lock to the right spot. Better aligning macro. Add a "continue" that should have been there when filename is seeked over during load. Change-Id: Ib0c517a7b77d3ba0dd6530ddef19bed62116e4a8
2017-02-13Tweak a few tagcache things.Michael Sevakis
* Take out pointless dircache_search; nothing can be reconstructed with the given info in find_entry_ram(); don't even try there. Path AND index id must be known. Work it out later. * Timed yield must be far more often than once every 1/4 second (?!) * Do better the memory-remaining checks for ramcache load. * Root separator mustn't be doubled up when searching files. Change-Id: I091813f4495f3bd0d0c4672bc674df52343b3e48
2017-02-10Fix warnings and errors from 7373cf5Michael Sevakis
dircache.h has to be included no matter what. Change-Id: Ib9ae4277615e8573b931d42fc6f1b3681d898bca
2017-02-10Restore dircache hookup in the database ramcache.Michael Sevakis
Do a few other changes to dircache and file code flags to accomodate its demands. Change-Id: I4742a54e8cfbe4d8b9cffb75faaf920dd907cf8a
2017-02-04Fix dangerous castsAmaury Pouly
On Windows 64-bit, the size of long is 32-bit, thus any pointer to long cast is not valid. In any case, one should use intptr_t and ptrdiff_t when casting to integers. This commit attempts to fix all instances reported by GCC. When relevant, I replaced code by the macros PTR_ADD, ALIGN_UP from system.h Change-Id: I2273b0e8465d3c4689824717ed5afa5ed238a2dc
2017-01-17Reenable database ramcache and playlist dircacheMichael Sevakis
Playlist dircache references should be back in working order. Reenabling dircache references in the database ramcache is not yet done as it requires quite a bit of rework. Otherwise, the database in RAM is functional again. Some buffer compatibility changes have been made for database commit because the dircache buffer can no longer be stolen, only freed by an API call. Change-Id: Ib57c3e98cb23e798d4439e9da7ebd73826e733a4
2014-08-30Rewrite filesystem code (WIP)Michael Sevakis
This patch redoes the filesystem code from the FAT driver up to the clipboard code in onplay.c. Not every aspect of this is finished therefore it is still "WIP". I don't wish to do too much at once (haha!). What is left to do is get dircache back in the sim and find an implementation for the dircache indicies in the tagcache and playlist code or do something else that has the same benefit. Leaving these out for now does not make anything unusable. All the basics are done. Phone app code should probably get vetted (and app path handling just plain rewritten as environment expansions); the SDL app and Android run well. Main things addressed: 1) Thread safety: There is none right now in the trunk code. Most of what currently works is luck when multiple threads are involved or multiple descriptors to the same file are open. 2) POSIX compliance: Many of the functions behave nothing like their counterparts on a host system. This leads to inconsistent code or very different behavior from native to hosted. One huge offender was rename(). Going point by point would fill a book. 3) Actual running RAM usage: Many targets will use less RAM and less stack space (some more RAM because I upped the number of cache buffers for large memory). There's very little memory lying fallow in rarely-used areas (see 'Key core changes' below). Also, all targets may open the same number of directory streams whereas before those with less than 8MB RAM were limited to 8, not 12 implying those targets will save slightly less. 4) Performance: The test_disk plugin shows markedly improved performance, particularly in the area of (uncached) directory scanning, due partly to more optimal directory reading and to a better sector cache algorithm. Uncached times tend to be better while there is a bit of a slowdown in dircache due to it being a bit heavier of an implementation. It's not noticeable by a human as far as I can say. Key core changes: 1) Files and directories share core code and data structures. 2) The filesystem code knows which descriptors refer to same file. This ensures that changes from one stream are appropriately reflected in every open descriptor for that file (fileobj_mgr.c). 3) File and directory cache buffers are borrowed from the main sector cache. This means that when they are not in use by a file, they are not wasted, but used for the cache. Most of the time, only a few of them are needed. It also means that adding more file and directory handles is less expensive. All one must do in ensure a large enough cache to borrow from. 4) Relative path components are supported and the namespace is unified. It does not support full relative paths to an implied current directory; what is does support is use of "." and "..". Adding the former would not be very difficult. The namespace is unified in the sense that volumes may be specified several times along with relative parts, e.g.: "/<0>/foo/../../<1>/bar" :<=> "/<1>/bar". 5) Stack usage is down due to sharing of data, static allocation and less duplication of strings on the stack. This requires more serialization than I would like but since the number of threads is limited to a low number, the tradoff in favor of the stack seems reasonable. 6) Separates and heirarchicalizes (sic) the SIM and APP filesystem code. SIM path and volume handling is just like the target. Some aspects of the APP file code get more straightforward (e.g. no path hashing is needed). Dircache: Deserves its own section. Dircache is new but pays homage to the old. The old one was not compatible and so it, since it got redone, does all the stuff it always should have done such as: 1) It may be update and used at any time during the build process. No longer has one to wait for it to finish building to do basic file management (create, remove, rename, etc.). 2) It does not need to be either fully scanned or completely disabled; it can be incomplete (i.e. overfilled, missing paths), still be of benefit and be correct. 3) Handles mounting and dismounting of individual volumes which means a full rebuild is not needed just because you pop a new SD card in the slot. Now, because it reuses its freed entry data, may rebuild only that volume. 4) Much more fundamental to the file code. When it is built, it is the keeper of the master file list whether enabled or not ("disabled" is just a state of the cache). Its must always to ready to be started and bind all streams opened prior to being enabled. 5) Maintains any short filenames in OEM format which means that it does not need to be rebuilt when changing the default codepage. Miscellaneous Compatibility: 1) Update any other code that would otherwise not work such as the hotswap mounting code in various card drivers. 2) File management: Clipboard needed updating because of the behavioral changes. Still needs a little more work on some finer points. 3) Remove now-obsolete functionality such as the mutex's "no preempt" flag (which was only for the prior FAT driver). 4) struct dirinfo uses time_t rather than raw FAT directory entry time fields. I plan to follow up on genericizing everything there (i.e. no FAT attributes). 5) unicode.c needed some redoing so that the file code does not try try to load codepages during a scan, which is actually a problem with the current code. The default codepage, if any is required, is now kept in RAM separarately (bufalloced) from codepages specified to iso_decode() (which must not be bufalloced because the conversion may be done by playback threads). Brings with it some additional reusable core code: 1) Revised file functions: Reusable code that does things such as safe path concatenation and parsing without buffer limitations or data duplication. Variants that copy or alter the input path may be based off these. To do: 1) Put dircache functionality back in the sim. Treating it internally as a different kind of file system seems the best approach at this time. 2) Restore use of dircache indexes in the playlist and database or something effectively the same. Since the cache doesn't have to be complete in order to be used, not getting a hit on the cache doesn't unambiguously say if the path exists or not. Change-Id: Ia30f3082a136253e3a0eae0784e3091d138915c8 Reviewed-on: http://gerrit.rockbox.org/566 Reviewed-by: Michael Sevakis <jethead71@rockbox.org> Tested: Michael Sevakis <jethead71@rockbox.org>
2014-03-14events: Rework event subsystem (add_event, send_event) to be more versatile.Thomas Martitz
add_event_ex is added that takes an extra user_data pointer. This pointer is passed to the callback (add_event and add_event_ex have slightly different callbacks types). All callbacks also get the event id passed. Events added with add_event_ex must be removed with remove_event_ex because the user_data pointer must match in addition to the callback pointer. On the other add_event is simplified to omit the oneshort parameter which was almost always false (still there with add_event_ex). As a side effect the ata_idle_notify callbacks are changed as well, they do not take a data parameter anymore which was always NULL anyway. This commit also adds some documentation to events.h Change-Id: I13e29a0f88ef908f175b376d83550f9e0231f772
2014-03-10Implement time-based resume and playback start.Michael Sevakis
This complements offset-based resume and playback start funcionality. The implementation is global on both HWCODEC and SWCODEC. Basically, if either the specified elapsed or offset are non-zero, it indicates a mid-track resume. To resume by time only, set elapsed to nonzero and offset to zero. To resume by offset only, set offset to nonzero and elapsed to zero. Which one the codec uses and which has priority is up to the codec; however, using an elapsed time covers more cases: * Codecs not able to use an offset such as VGM or other atomic formats * Starting playback at a nonzero elapsed time from a source that contains no offset, such as a cuesheet The change re-versions pretty much everything from tagcache to nvram. Change-Id: Ic7aebb24e99a03ae99585c5e236eba960d163f38 Reviewed-on: http://gerrit.rockbox.org/516 Reviewed-by: Michael Sevakis <jethead71@rockbox.org> Tested: Michael Sevakis <jethead71@rockbox.org>
2013-01-15Fix possible readlink() buffer overflowThomas Jarosch
readlink() might return the full size of the target buffer and we write a '\0' to the returned length offset. cppecheck reported: [rockbox/apps/tagcache.c:4335]: (warning, inconclusive) readlink() might return the full size of 'target'. Lower the supplied size by one. (the check was actually written by me) Change-Id: Ibb42f732aa42c38bb6cb92cdccd3e6a0d3aa9b9f
2012-07-31Fix database tool.Thomas Martitz
It was also broken functionally, probably since a while.So restore the functionality. Run it on the dap, the tcd files will be placed into .rockbox folder. Change-Id: Id7a6ce4389dfaf99799258902be80d630af0601c
2012-07-30Database: Support for multiple search roots.Thomas Martitz
Support multiple roots (i.e. scan folders) using the new folder_select() function. Amaximum of 12 dirs can be selected, however the setting (i.e. the entire folder list) cannot be longer than 80 chars. The setting works similar to the autoresume dirs: Directories are seperated by colons, e.g. "/Music:/Podcasts". Default is "/sdcard" on android, "/" on all other targets. Changes are made to the multiple-root logic in tagcache.c. This is to a) provide the logic at all on native targets and b) support more than one root before the scan even starts (until now it was used to add roots during scan due to symlinks). Change-Id: I64992c0678324536e8e64cd4427c8abbd8e8b39e
2011-12-17Add const to global pointers to strings.Boris Gjenero
When a global pointer is not declared as constant, gcc will put it in memory. Getting the address of the string it points to requires loading the address of the pointer and then loading the pointer. When the pointer is declared constant, the address of the string is loaded directly. git-svn-id: svn://svn.rockbox.org/rockbox/trunk@31345 a1c6a512-1295-4272-9138-f99709370657
2011-12-14FS#12378 : Remove various unused code, and comment out some unused code and ↵Boris Gjenero
data for reference or future use. git-svn-id: svn://svn.rockbox.org/rockbox/trunk@31256 a1c6a512-1295-4272-9138-f99709370657
2011-11-17Simulate usb plugging on the sim better using sim_tasks.Thomas Martitz
Now all threads need to ack the connection like on real target, dircache is unloaded and playback stops accordingly. git-svn-id: svn://svn.rockbox.org/rockbox/trunk@31009 a1c6a512-1295-4272-9138-f99709370657
2011-08-30Dircache: Allow dircache to be enabled without reboot.Thomas Martitz
Also add two dircache function, one of which does what dircache_disable() did previously as this now also frees the dircache buffer. git-svn-id: svn://svn.rockbox.org/rockbox/trunk@30393 a1c6a512-1295-4272-9138-f99709370657
2011-08-30Fix h100 red and other warnings.Thomas Martitz
git-svn-id: svn://svn.rockbox.org/rockbox/trunk@30384 a1c6a512-1295-4272-9138-f99709370657
2011-08-30GSoC/Buflib: Enable compaction in buflib.Thomas Martitz
This enables the ability to allocate (and free) memory dynamically without fragmentation, through compaction. This means allocations can move and fragmentation be reduced. Most changes are preparing Rockbox for this, which many times means adding a move callback which can temporarily disable movement when the corresponding code is in a critical section. For now, the audio buffer allocation has a central role, because it's the one having allocated most. This buffer is able to shrink itself, for which it needs to stop playback for a very short moment. For this, audio_buffer_available() returns the size of the audio buffer which can possibly be used by other allocations because the audio buffer can shrink. lastfm scrobbling and timestretch can now be toggled at runtime without requiring a reboot. git-svn-id: svn://svn.rockbox.org/rockbox/trunk@30381 a1c6a512-1295-4272-9138-f99709370657
2011-08-30GSoC/Buflib: Add buflib memory alocator to the core.Thomas Martitz
The buflib memory allocator is handle based and can free and compact, move or resize memory on demand. This allows to effeciently allocate memory dynamically without an MMU, by avoiding fragmentation through memory compaction. This patch adds the buflib library to the core, along with convinience wrappers to omit the context parameter. Compaction is not yet enabled, but will be in a later patch. Therefore, this acts as a replacement for buffer_alloc/buffer_get_buffer() with the benifit of a debug menu. See buflib.h for some API documentation. git-svn-id: svn://svn.rockbox.org/rockbox/trunk@30380 a1c6a512-1295-4272-9138-f99709370657
2011-08-25Fix file descriptor leak on errorThomas Jarosch
Credit goes to "cppcheck" again. git-svn-id: svn://svn.rockbox.org/rockbox/trunk@30350 a1c6a512-1295-4272-9138-f99709370657
2011-08-14Fix tempbuf assignment in tagcache and bump plugin API/ABI for r30308.Thomas Martitz
git-svn-id: svn://svn.rockbox.org/rockbox/trunk@30314 a1c6a512-1295-4272-9138-f99709370657
2011-08-14GSoC/Buflib: Replace all direct accesses to audiobuf with buffer API functions.Thomas Martitz
Namely, introduce buffer_get_buffer() and buffer_release_buffer(). buffer_get_buffer() aquires all available and grabs a lock, attempting to call buffer_alloc() or buffer_get_buffer() while this lock is locked will cause a panicf() (doesn't actually happen, but is for debugging purpose). buffer_release_buffer() unlocks that lock and can additionally increment the audiobuf buffer to make an allocation. Pass 0 to only unlock if buffer was used temporarily only. buffer_available() is a replacement function to query audiobuflen, i.e. what's left in the buffer. Buffer init is moved up in the init chain and handles ipodvideo64mb internally. Further changes happened to mp3data.c and talk.c as to not call the above API functions, but get the buffer from callers. The caller is the audio system which has the buffer lock while mp3data.c and talk mess with the buffer. mpeg.c now implements some buffer related functions of playback.h, especially audio_get_buffer(), allowing to reduce #ifdef hell a tiny bit. audiobuf and audiobufend are local to buffer.c now. git-svn-id: svn://svn.rockbox.org/rockbox/trunk@30308 a1c6a512-1295-4272-9138-f99709370657
2011-08-04Database: Bug fix: The filename seek index is invalid if FLAG_DIRCACHEMichael Hohmuth
is set on a database entry and the dircache went offline. In this case, retrieve() and get_next() need to abort and take the ramcache offline as well. git-svn-id: svn://svn.rockbox.org/rockbox/trunk@30255 a1c6a512-1295-4272-9138-f99709370657
2011-08-04Database: find_entry_ram: Avoid dircache use if it's not readyMichael Hohmuth
git-svn-id: svn://svn.rockbox.org/rockbox/trunk@30254 a1c6a512-1295-4272-9138-f99709370657
2011-08-04Database: Fix memory-area bounds checking during database reload.Michael Hohmuth
Check free space before reading new data from disk, and do not forget to account for the RAM-cache header. git-svn-id: svn://svn.rockbox.org/rockbox/trunk@30246 a1c6a512-1295-4272-9138-f99709370657
2011-08-02Database: Fix to support case-sensitive file systems containing audioMichael Hohmuth
files with names differing only in capitalization. Thanks to Slasheri for the fix. git-svn-id: svn://svn.rockbox.org/rockbox/trunk@30241 a1c6a512-1295-4272-9138-f99709370657
2011-08-02Tagcache: Rename global static variable for better readability.Thomas Martitz
git-svn-id: svn://svn.rockbox.org/rockbox/trunk@30239 a1c6a512-1295-4272-9138-f99709370657
2011-07-31FS#12132 patch 6, part 2: tagnavi.config: Add support for "basename"Michael Hohmuth
in formats and conditions. The basename is the non-directory part of a file's pathname. git-svn-id: svn://svn.rockbox.org/rockbox/trunk@30230 a1c6a512-1295-4272-9138-f99709370657
2011-07-01Allocate indices directly in ramcache header. Add version code toMiika Pekkarinen
tagcache state dump header. Debug: hdr-structure corruption might happen after rebuilding of dircache. git-svn-id: svn://svn.rockbox.org/rockbox/trunk@30109 a1c6a512-1295-4272-9138-f99709370657
2011-06-23Fixed a regression caused in r30021: tagnavi_custom.config parsingMiika Pekkarinen
fails if lines ended with <CR><LF> sequence. git-svn-id: svn://svn.rockbox.org/rockbox/trunk@30058 a1c6a512-1295-4272-9138-f99709370657
2011-06-20Dircache: Don't expose struct dircache_entry and pointers into the cache, ↵Thomas Martitz
use IDs instead. Only integer IDs are exposed from dircache with this. This way the cache is isolated from other modules. This is needed for my buflib gsoc project. git-svn-id: svn://svn.rockbox.org/rockbox/trunk@30038 a1c6a512-1295-4272-9138-f99709370657
2011-06-20Dircache: Return the size of the result string in dircache_copy_path() so ↵Thomas Martitz
that callers don't need to call strlen on it. git-svn-id: svn://svn.rockbox.org/rockbox/trunk@30034 a1c6a512-1295-4272-9138-f99709370657
2011-06-20Fixed a changelog export/import problem with tagcache where <CR> or <LF>Miika Pekkarinen
characters in a tag could cause the parser fail to import a track statistics correctly. Now line feeds are escaped properly and carriage returns ignored on import. git-svn-id: svn://svn.rockbox.org/rockbox/trunk@30021 a1c6a512-1295-4272-9138-f99709370657
2011-06-19Removed redundant copy of tagcache master_header structure stored inMiika Pekkarinen
memory. One of these headers (current_tcmh) was not loaded in when tagcache state was initialized from a hibernated state file (flashed H1xx targets). That caused internal serial number to start always from zero, rendering "recently played tracks" query not working as expected. git-svn-id: svn://svn.rockbox.org/rockbox/trunk@30016 a1c6a512-1295-4272-9138-f99709370657
2011-06-06FS#12107: Remove track-number guessing code.Michael Hohmuth
git-svn-id: svn://svn.rockbox.org/rockbox/trunk@29982 a1c6a512-1295-4272-9138-f99709370657
2011-06-06FS#12132 patch 5: check_clauses: Optimizations.Michael Hohmuth
* There's no need to reset the buffer argument to check_against_clauses to 0 when it's not needed. We can always initialize str = buf and only change it on a RAM search hit. * Do not memset buffer to 0 -- it's sufficient to make sure the retrieved tag string is zero-terminated. * Factor out a call to check_virtual_tags from two branches of an if statement. git-svn-id: svn://svn.rockbox.org/rockbox/trunk@29981 a1c6a512-1295-4272-9138-f99709370657
2011-06-06FS#12132 patch 3: Rename read_numeric_tag to find_tag, as it is indeedMichael Hohmuth
used for nonnumeric tags as well. Optimization for the nonnumeric case: Only numerics are ever updated, so there's no need to scan the command queue for updates to nonnumeric tags. git-svn-id: svn://svn.rockbox.org/rockbox/trunk@29979 a1c6a512-1295-4272-9138-f99709370657
2011-06-06FS#12132 patch 2: add_uniqbuf: Fix confusing indentation in if conditionMichael Hohmuth
git-svn-id: svn://svn.rockbox.org/rockbox/trunk@29978 a1c6a512-1295-4272-9138-f99709370657
2011-05-11FS#12076 - DB stats resurrection: If the filename was changed, requireMichael Hohmuth
all three of album, artist, and title tags to match for resurrection to take place. git-svn-id: svn://svn.rockbox.org/rockbox/trunk@29858 a1c6a512-1295-4272-9138-f99709370657
2011-05-10tagnavi.config: Add a logical-OR operator ("|") for tagnavi conditionals.Michael Hohmuth
Logical-AND ("&") always takes precedence over logical-OR. (Parentheses are not supported.) Fight binsize increase by storing some common expressions in local variables. This avoids repeated reevaluations involving memory accesses. check_clauses: Fail clause (return false) in case of errors (tag too long, DB entry deleted). git-svn-id: svn://svn.rockbox.org/rockbox/trunk@29851 a1c6a512-1295-4272-9138-f99709370657
2011-05-03Fix even more 'variable set but not used' warnings reported from GCC 4.6.0.Andree Buschmann
git-svn-id: svn://svn.rockbox.org/rockbox/trunk@29819 a1c6a512-1295-4272-9138-f99709370657
2011-04-15Do not attempt to remove the temporary database twice during a database update.Michael Hohmuth
git-svn-id: svn://svn.rockbox.org/rockbox/trunk@29716 a1c6a512-1295-4272-9138-f99709370657
2011-03-05Follow symlink dirs on UI simulatorFred Bauer
git-svn-id: svn://svn.rockbox.org/rockbox/trunk@29530 a1c6a512-1295-4272-9138-f99709370657
2011-02-28DB import: Correctly import previously exported resume offsetsMichael Hohmuth
git-svn-id: svn://svn.rockbox.org/rockbox/trunk@29469 a1c6a512-1295-4272-9138-f99709370657
2011-02-27Fix database import and exportMichael Hohmuth
Correct an omission that happened when the DB schema was changed last time. Add a comment to enum tag_type to prevent this from happening in the future. git-svn-id: svn://svn.rockbox.org/rockbox/trunk@29435 a1c6a512-1295-4272-9138-f99709370657
2011-02-16Fix buffer size used for realpath() callThomas Jarosch
Thanks to kugel for pointing this out. git-svn-id: svn://svn.rockbox.org/rockbox/trunk@29320 a1c6a512-1295-4272-9138-f99709370657
2011-02-16No need to cast away const. Thanks to sideralThomas Jarosch
git-svn-id: svn://svn.rockbox.org/rockbox/trunk@29318 a1c6a512-1295-4272-9138-f99709370657
2011-02-16RaaA: Fix tagcache filename lookups for paths containing symbolic linksThomas Jarosch
git-svn-id: svn://svn.rockbox.org/rockbox/trunk@29317 a1c6a512-1295-4272-9138-f99709370657