<< >>
justin = { main feed , music , code , askjf , pubkey };recent comments
search
Searching for 'proved' in (articles)... [back to index]

February 2, 2017
macOS screen updating, 2017 edition

TL;DR: Retina iMac (4k/5k) owners can greatly improve the graphics performance of many applications (including REAPER) by setting the color profile (in System Preferences, Displays, Color tab) to "Generic RGB" or "Adobe RGB." (and restarting REAPER and/or other applications being tested)

I previously wrote in mid-2014 about the state of blitting bitmaps to screen on modern OS X (now macOS) versions. Since then, Apple has released new hardware (including Retina iMacs) and a couple of new macOS versions.

Much of that article is still useful today, but I made a mistake in the second update:

    OK, if you provide a bitmap that is twice the size of the drawing rect, you can avoid argb32_image_mark_RGBXX, and get the Retina display to update in about 5-7ms, which is a good improvement (but by no means impressive, given how powerful this machine is). I made a very simple software scaler (that turns each pixel into 4), and it uses very little CPU.
While this was helpful (and did decrease the amount of time spent blitting), it was wrong in that the reason for the faster blit was that the system was parallelizing the blit with multiple cores. So, it was faster, but it also used more CPU (and was generally wasteful).

I discovered this because I've been researching how to improve REAPER's graphic performance on the iMac 5k in particular, so I started benchmarking. This time around, I figured I should measure how many screen pixels are updated and divide that by how long it takes. Some results, based on my memory (I'm not going to rerun them for this article, laziness).

Initial version (REAPER 5.32 state, using the retina hack described above, public WDL as of today):
  • old C2D iMac, 10.6: 350MPix/sec
  • mid-2012 RMBP 15", 10.12, Thunderbolt display (non-retina): 1500MPix/sec
  • mid-2012 RMBP 15", 10.12, built-in display (retina): 800MPix/sec
  • late-2015 Retina iMac 5k, 10.12: 192MPix/sec
The one that really jumped out at me was the Retina iMac 5k -- it's a quarter of the speed of the RMBP! WTF. We'll get to that later.

After I realized the hack above was actually doing more work (thank you, Xcode instrumentation), I did some more experiments, avoiding the hack, and found that in the newer SDKs there are kCGImageByteOrderXYZ flags (I don't believe it was in previous SDKs), and found that these alised to KCGBitmapByteOrderXYZ, and that when using kCGBitmapByteOrder32Host with the pixel format for CGImageCreate()/etc, it would speed things up. With retina hack removed:
  • mid-2012 RMBP 15", 10.12, built-in display (retina): 300MPix/sec
  • late-2015 Retina iMac 5k, 10.12: 152MPix/sec
With retina hack removed and byte order set to host:
  • old C2D iMac, 10.6: 350MPix/sec
  • mid-2012 RMBP 15", 10.12, Thunderbolt display (non-retina): 1500MPix/sec
  • mid-2012 RMBP 15", 10.12, built-in display (retina): 720MPix/sec
  • late-2015 Retina iMac 5k, 10.12: 200MPix/sec
The non-retina displays might have changed slightly, but it was insignificant. So, by setting the byte order to native, we get the Retina MBP close to the level of performance of the hack, which isn't great but is serviceable, and at least the CPU use is decreased. This also has the benefit (drawback?) of making the byte-order of pixels the same on macOS/Intel and win32, which will take some more attention (and a lot of testing).

From profiling and looking at the code, this blit performance could easily be improved by Apple -- the inner loop where most time is being spent does a lot more than it needs to. Come on Apple, make us happy. Details offered on request.

Of course, this really doesn't do anything for the iMac 5k -- 200MPix/sec is *TERRIBLE*. The full screen is 15 megapixels, so at most that gets you around 13fps, and that's at 100% CPU use. After some more profiling, I found that the function chewing the most CPU ended in "64". Then it hit me -- was this display running in 16 bits per channel? A quick google search later, it was clear: the Retina iMacs have 10-bit displays, and you can run them in 10 bits per channel, which means 64 bits per pixel. macOS is converting all of our pixels to 64 bits per pixel (I should also mention that it seems to be doing a very slow job of it). Luckily, changing the color profile (in system preferences, displays) to "Generic RGB" or similar disables this, and it gets the ~800MPix/sec level of performance similar to the RMBP, which is at least tolerable.

Sorry for the long wordy mess above, I'm posting it here so that google finds it and anybody looking into why their software is slow on macOS 10.11 or 10.12 on retina imacs have some explanation.

Also please please please Apple optimize CGContextDrawImage()! I'm drawing an image with no alpha channel and no interpolation and no blend mode and the inner loop is checking each pixel to see if the alpha is 255? I mean wtf. You can do better. Hell, you've done way better. All that "new" Retina code needs optimizing!

Update a few hours later:
Fixing various issues with the updated byte-ordering, CoreText produces quite different output for CGBitmapContexts created with different byte orderings:


Hmph! Not sure which one is "correct" there... hmm... If you use kCGImageAlphaPremultipliedFirst for the CGBitmapContext rather than kCGImageAlphaNoneFirst, then it looks closer to the original, maybe. ?

Also other caveat: NSBitmapImageRep can't seem to deal with the ARGB format either, so if you use that you need to manually bswap the pixels...

Update (2019): SolvedWorked around most of this issue by using Metal, read here.

4 Comments


December 14, 2013
a month later, midi2osc becomes OSCII-bot

midi2osc, as mentioned in the last post, got some updates which made its name obsolete, particularly the ability to send MIDI and receive OSC, so it has now been renamed OSCII-bot. Other mildly interesting updates:

  • OSX version
  • Can load multiple scripts, which run independently but can share hardware
  • Better string syntax (normal quotes rather than silly {} etc), user strings identified by values 0..1023
  • Better string manipulation APIs (sprintf(), strcpy(), match(), etc).
  • match() and oscmatch(), which can be used for simple regular expressions with the ability to extract values
  • Ability to detect stale devices and reopen them
  • Scripts can output text to the newly resizeable console, including basic terminal emualtion (\r, and \33[2J for clear)
  • Vastly improved icon
I'll probably get around to putting it on cockos.com soon enough, but for now a new build is here. Read the readme.txt before using for instructions.

The thing I'm most excited about, in this, is the creation of eel_strings.h, which is a framework for extending EEL2 (the scripting engine that powers JSFX, for one) to add string support. Adding support for strings to JSFX will be pretty straightforward, so we'll likely be doing that in the next few weeks. Fun stuff. Very few things are as satisfying as making fun programming languages to use...

8 Comments


April 11, 2010
Dear Intel,

I've just read this article about your experimental 48-core CPU. Can I mention that our software, REAPER is highly multi-core optimized, and we'd love to see how it runs on / can be improved for 48 cores?

Sincerely,

Justin Frankel
Cockos Incorporated

P.S. We have REAPER running on Linux, too. Don't tell our users though, or they'll start whining (you know how Linux users can be).



10 Comments


January 10, 2009
Autosong.ninjam.com

...now new and improved!

3 Comments


February 15, 2008
lock-in

Was reading an article on Bruce Schneier's blog about software lock-in, and found this quote intruiging:

    Economists Carl Shapiro and Hal Varian even proved that the value of a software company is the total lock-in. Here's the logic: Assume, for example, that you have 100 people in a company using MS Office at a cost of $500 each. If it cost the company less than $50,000 to switch to Open Office, they would. If it cost the company more than $50,000, Microsoft would increase its prices.
I absolutely despise artificial or intentional lock-in and am attempting to make a software company that tries (actively) to do as little as possible to lock people in-- nothing, that is, except making the best software possible. Yikes, there is a long shallow road ahead of us...

6 Comments


October 10, 2007
Radiohead "In Rainbows"

I'm digging it. A lot. Not as huge to me as some past Radiohead albums were, but then again I probably ruined some of the impact for myself seeing them play these songs live a few times first, whereas before I heard Kid A the first time I had never heard any of it. Not that I'm complaining-- seeing Radiohead live is absurdly fantastic...

October 10, 2007
REAPER 2.0 woohoo


After over a month in beta, we finally released 2.0 final. Woohoo. Here's the changelog (condensed but shown here so I can reflect on the lengthh of it):

  • added elastique Pro, Efficient, and SOLOIST as pitch shifter/time stretcher options
  • action: new actions to toggle/clear/set individual lock modes
  • action: actions to set take by index (1-9) active
  • action: "Take/Paste as takes in selected items"
  • action: "render items to new take" (which is like apply fx but without fx)
  • action: action to toggle item "preserve pitch when changing playrate"
  • automation: added option prefs/editing/"Automatically add/arm envelopes when tweaking parameters in write modes"
  • automation: autoadding vol/pan/playspeed envelopes autoresets trims to unity
  • automation: mute envelopes for tracks/sends (no UI integration for automation recording yet)
  • automation: fixed vol/pan/playspeed tooltips when in automation modes
  • compatibility: perf meter: fixed incorrect ram usage on w2k
  • compatibility: fixed a win2k text drawing gdi corruption issue
  • compatibility: fixed a win2k media explorer refreshing bug
  • compatibility: vertical zooming now flickery in WINE (since WINEs WM_SETREDRAW breaks things -- WINE developers, contact us)
  • defaults: made Take Lane viewing on by default
  • display: fixed bug with changing screen resolutions
  • editing: fixed cutting items in ripple all mode
  • editing: split items at loop selection selects only items in selection (not unsplit previously selected items)
  • editing: apply fx to new take now works on empty items
  • editing: better zoom from scrollbar when zoom set to center on mouse cursor
  • editing: enabled zoom out to see more than a few hours
  • editing: better envelope behavior in item moving and ripple editing
  • editing: fixed bug with slip editing items fudging automation
  • editing: shifting/nudging loop selection works better with time signatures
  • editing: main track view sub-pixel accuracy improvements
  • editing: ctrl+dragging loop selections when item left/right locked now works
  • editing: force selection to beat lengths now supports multiple time signatures
  • editing: new item lock modes (item edges, controls)
  • editing: better drag and drop file positioning in certain instances
  • fx: comment window is now modeless
  • fx: added vertical scrollbar to comment window
  • fx: you can now rename instances of effects to better describe their application
  • fx: action to build multichannel routing for the output of multichannel VSTis
  • fx: action to build 16 channels of midi routing for the current track
  • fx: vsts that have latency and send MIDI now can send ahead of time
  • fx: updated VST samplerate change calls for buggy plug-ins
  • fx: special case code for simulanalog VST plugins (aggressive denormal prevention)
  • fx: fixed alt+drag fx moving bugs
  • fx: fixed plug-in config window close order on quit (good for EmuX)
  • fx: better denormalization prevention methods used throughout
  • fx: faster offlining of plug-ins with large state data
  • fx: startup project loading now initializes audio device before loading plug-ins
  • fx: modifiers when drag and drop adding fx (shift=dont bring up config, ctrl=toggle floating of config)
  • fx: floating fx windows remember their positions when closed
  • fx: less showing of fx chain when "auto-float new fx" is on
  • FX: added JS: utility/bufsave, which lets you route feedback in fx chains easily
  • FX: added JS: utility/time_adjustment which allows delay/predelaying signal
  • FX: added JS: autopeakfilter for fun autowah type effects
  • FX: added some new loser JS fx
  • FX: JS PDC support for effects (pdc_bot_ch, pdc_top_ch, pdc_midi and pdc_delay to specify sample delay)
  • FX: JS shared memory (gmem[]) is now shared across all JS instances in reaper
  • FX: added JS play_state, play_position, and beat_position variables
  • FX: safer window class registration/unregistration in many Rea* fx
  • FX: reacomp/reaxcomp performance improvements
  • FX: better offscreen window checking for ReaNINJAM
  • FX: ReaTune now uses REAPERs pitch shift algorithms (elastique soloist is great for this)
  • FX: ReaTune added "click reduction" mode for SoundTouch and possibly other modes
  • FX: ReaTune manual mode ruler, mousewheel support
  • FX: ReaTune subdivision mode (to update at higher frequencies with larger window sizes)
  • FX: ReaPitch, new multi voice pitch shifter
  • FX: ReaEQ and ReaXComp now update undo states on add/remove of bands
  • installer: now allows selection of pitch shifters to install
  • keyboard: better handling of keystrokes when mouse captured
  • master track: can now have more than 2 channels
  • master track: can now control source channels/phase/volume/pan/etc of each hardware output independently
  • master track: better RMS metering, lots of RMS display options
  • master track: better pdc with anticipative rendering
  • master track: master fx chain no longer defaults to bypassed
  • master track: made solo/mute click modifiers not affect master mute/solo
  • master track: right click marquee works in master track (for envelopes etc)
  • master track: fixed undo with no master hardware outs sometimes adding in a default output
  • media explorer: added "Insert as takes in selected items"
  • media item properties: "choose new file" automatically updates take names
  • meters: better event light for record output (midi) mode
  • meters: updated track metering rounding to better pass synthetic tests
  • meters: better track meter clip indicator hit testing
  • meters: better metering for record output (midi) mode
  • metronome: better metronome countin for tempo changes
  • midi: MID file import can now import tempo maps
  • midi: better looking and faster midi peak drawing
  • midi: midi items are now treated as ticks/Quarter Note, except for items in old projects (which are still ticks/beat)
  • midi: fixed open copy of items sometimes dropping notes
  • midi: better sorting of noteoffs and allnoteoff messages
  • midi: reduced excess sending off allnoteoff loop markers
  • midi: fixed splitting items on notes producing 0 length notes
  • midi: fixed bug in fadein for midi items
  • midi: fixed extraneous notes at end of some items
  • midi editor: can now reflect project time signature changes
  • midi editor: grid/quantize are now fractions of whole notes, not of beats
  • midi editor: separate colortheme settings in prefs (including for piano keys etc)
  • midi editor: fixed bugs with play cursor and looped midi items
  • midi editor: better vertical scrollbar
  • midi editor: better focusing when opening/activating and switching modes
  • midi editor: internal cleanups, improved ctrl+select behavior
  • midi editor: pass through to main window keyboard action
  • midi editor: updates to list editor play cursor, better list editor sorting
  • midi editor: list editor note properties sets focus depending on which column the mouse was on
  • midi editor: better focusing when opening/activating and switching modes
  • midi editor: moving notes now uses both absolute and relative snapping
  • mixer: optional FX and Send views (configurable via the mixer menu)
  • mixer: more settings are now stored in the project
  • mixer: options are now assignable actions
  • mixer: new FX/send views are themeable (mcp_fxlist_norm/byp/off/empty, mcp_master_fxlist_ too)
  • mixer: (theme images): mcp_sendlist_knob.png, mcp_sendlist_meter.png, mcp_*list_arrows.png
  • mixer: fixed incorrect minimum height on nonstandard display DPIs
  • Monkeys Audio: fix for Unicode files, fixes for offline support
  • Monkeys Audio: fixed mem leak, fixed 24 bit stereo mode, optimizations
  • Monkeys Audio: now uses asynchronous disk reads if set
  • option: options to not show peaks for muted tracks/items, or non-selected tracks
  • option: added working set configuration in prefs/general/advanced
  • performance: portions of REAPER are now compiled with the Intel C++ compiler
  • performance: faster zoomed-in peaks display
  • performance: fixed muted folder tracks still running fx
  • performance: updated on-stop behavior to keep audio thread locked for less time
  • performance: UAD synchronous fx multiprocessing support
  • performance: improved anticipative fx processing on looped playback
  • performance: renamed fx renderahead "anticipative FX processing"
  • performance: added new "Synchronous multiprocessing" option, which allows multiprocessing on input monitoring, better UAD multiprocessing, etc
  • performance: per-item pitch shifters are now freed when they are no longer needed or media set offline
  • performance: fixed silence-at-end-of-rendering issue with asynchronous writes enabled
  • performance: more robust asynchronous disk writes
  • performance: fixed SMP rendering glitches
  • performance: fixed cpu munch when stopping at end of project
  • performance: mono items pitch shifted are now processed in mono (big speedup)
  • performance: reduced cpu use of empty tracks
  • perf meter: graph shows cur/avg, range, action to reset graph
  • project directory cleanup: now defaults to sending items to recycle bin rather than deleting
  • reamote: possible fix for Nebula plugins
  • reamote: fixed crash when invalid data received on certain message types
  • reamote: fixed support for larger config packets
  • routing: fixed labelling on hardware outs after adding routing
  • routing: renamed send type "Post-FX" to "Post-FX (V1 deprecated)", added a new, better "Post-FX" mode
  • routing: fixed i/o windows open when adjusting/removing routing bugs
  • routing: better rearoute labelling all around when audio device closed
  • screensets: added keyboard shortcut column and button to edit shortcuts
  • screensets: autosave wont save anymore when switching to same screenset
  • screensets: added auto saving option
  • screensets: fixed docker issues
  • screensets: added mixer flags saving
  • screensets: added last focused window state saving
  • tempo map: overhaul: simple tempo changes no longer force a new measure
  • tempo map: improved tempo editing behavior when editing tempo changes and time sig changes
  • tempo map: lengths calculated across timesig markers now use the time signature at the start to determine measure length
  • tempo map: more accurate tempo envelope saving/restoring
  • tempo map: small fades are no longer adjusted by tempo changes
  • theming: faster drawing, faster mouseover updating
  • theming: background for faders, window backgrounds, name backgrounds, etc support pink line for unstretch areas
  • theming: docker is now independently colorthemeable
  • theming: new default colortheme (by WhiteTie!)
  • theming: support for when path to theme dir changes on diff systems
  • theming: advanced faders can have zero line set
  • theming: configurable fonts for track panels / volpan labels etc
  • theming: track_fxempty_h and track_fxempty_v
  • theming: window drawing improvements (less screen corruption issues)
  • theming: fixed stopped resize of transport drawing issues
  • theming: fixed some playspeed automation refresh issues
  • theming: fixed some dark custom color issues with advanced themes
  • theming: better background image edge scaling when compressed to small spaces
  • theming: classic theme color tweaks (for mcp send/fx list)
  • theming: bg tinting for track labels in advanced themes
  • ui: most modal dialog boxes now restore window focus on close
  • ui: added option for gap between items on adjacent tracks (defaults to 4px)
  • ui: fixed non-fancy peaks display on muted items/inactive takes
  • ui: better envelope spacing (small gap between envelopes)
  • ui: better item loop indicator drawing
  • ui: fixed edge loop indicators on some items
  • ui: fixed zero line drawing issue on items
  • ui: fixed peaks display at end of heavily looped items at certain zoom levels
  • ui: better record mode display for midi overdub modes
  • ui: fader ctrl+precision modes hide mouse cursor
  • ui: better spacing for transport status state
  • ui: better volume fader ganging at extremes
  • ui: grid/snap boxes will now correctly display smaller fractions
  • fixed some tiny PCM-floating point import/export precision issues
  • better external midi editor support (fixes, open copy in external editor creates .mid file)
  • faster dB unit conversion throughout
  • demo song: updated mix
  • new about screen



6 Comments


May 29, 2007
I got a guitar amp that I LOVE

For those of you dedicated bored enough to listen to our Recordings, you may have noticed that the guitar sounds we've been getting have improved dramatically.

I attribute this to my newish custom built guitar amp, from Underhill. HOLY CRAP. This thing rocks. It's just so much more fun to play than anything else I've played on. It's similar to a JCM 800 with a bunch of goodies, and something special that comes from the love of being handmade by someone who really cares. Wow! Before this I thought playing using the CrusFX with modelling was adequate! No more (anybody want to buy a CrusFX? OK I should probably keep it even though it hasn't been powered on in ages)....

Here's a little clip with some clean and then some loud in the middle: Song about Flying Cars and Unicorns.

8 Comments


May 25, 2007
REAPER in Sound On Sound

Martin Walker was nice enough to review REAPER in the UK magazine Sound On Sound.. I think it's a great review and am very proud to see it. Here's a quote from the end:

    ...unless you are 100 percent happy with your current MIDI + Audio sequencer, you'd be a fool not to give REAPER a try. Don't be fooled by the tiny 2MB download: this is a powerful application that does a huge amount, is being improved on a daily basis, and already does some things better than most of the competition.

I put this quote and some others on the REAPER Reviews Page. Woo.

May 25, 2007
you care a lot about your perception when it's all you got


Today I saw a Technology Review article on eJamming, which is a service for people to jam together online. I have tried eJamming a few times, and haven't been terribly impressed (and yes I am biassed towards NINJAM for obvious reasons), but I thought I'd just pick apart what bugs me about this article (and likewise the whole situation).

    First, the eJamming software decreases the file sizes sent over the network. To do this, the company's engineers developed their own compression and decompression algorithms that shrink the file size, yet still maintain an audio quality higher than MP3, a common compression scheme, says Glueckman.
OK so shrinking the size of data does help latency, but it's not the biggest part. What bugs me here is that A) they appear to be using Speex for their audio side, which is an open format, and doesn't scale to high qualities. Do they really need to claim that they developed their own codec? OK so maybe they did develop their own codec-- why isn't this news? If it's so much better than OGG or MP3 or AAC, how come we haven't heard of this?

    Second, each musician is directly connected with the other musicians in a jam session, instead of being routed through a server. This peer-to-peer configuration "results in a lower latency by routing the audio stream directly to your jam mates rather than, on average, doubling that transport latency by directing the audio stream through a remote server," says Bill Redmann, chief technology officer of eJamming.
Wow! Connecting clients to clients! Peer to peer! This is so unique!

But what the article fails to mention here is that while this approach does reduce latency, it also increases the amount of bandwidth requires by each host exponentially. Their stuff is limited to 4 people I believe, but if you have 4 people, each client needs to send its stream to EVERYBODY else, so if they run a 50kbps channel (though it appears they're using a much lower bitrate codec anywa), that'd be 150kbps of upstream for a 4 person jam. Which isn't that bad, but it doesnt scale well and definitely requires some decent broadband.

The following quote, however, is what really bugs me:

    The company is promising to reduce the delay experienced over the network to, at most, hundreds of milliseconds (depending on upload speed and geographic distance between musicians)--a delay to which, Glueckman says, most musicians can adjust with practice.

Hundreds of milliseconds?! Are you kidding me? This is unusable. Their patented "delay monitoring of the local signal to sync with the remote" even makes it worse. I tried it with Christophe, who is on the same ISP and less than 10 miles away, and the latency was very noticeable and made it difficult to play anything remotely complex.

Imagine trying to play synchronized with someone at the other end of a football field. ugh.

Granted NINJAM's solutions aren't perfect either, but I find the increased quality and overall experience to be far superior. And the newer voice chat and session modes are damn usable.

OK OK so I didn't mean to turn this into a big eJamming vs NINJAM thing.. I just am irritated with these people being all "look we're soo innovative" and having lofty claims, yet when we use their software it's miserable (and I didn't even go into how badly constructed their application is-- application development isn't an easy thing, and in this case it definitely shows).

May 25, 2007
mac poo


I hate Xcode. Too bad I have a ton to get done using it.

4 Comments


April 21, 2007
NINJAM has been getting some love

Been updating NINJAM, after integrating it into REAPER. The REAPER version has a much revised UI, and now supports sending stereo streams, a low latency voice chat mode, and will soon get a fancy "Session mode" that will ease long distance collaboration on projects.

Also updated the NINJAM AutoSong program and web site.. Been listening to playlists of random songs cut, and they're getting very good! I improved the program that does the cutting/mixing last week too.. Uses smarter volume detection and mixing logic, also does nice fadeouts.

Been having fun improving various aspects of REAPER that have been lacking.. The biggest one coming that's almost done is making loop selections and playback and recording sub-sample accurate. In previous versions looping was only accurate to the audio block. Soon it will be VERY accurate..



2 Comments




Search comments - Ignore HTML tags
search : rss : recent comments : Copyright © 2024 Justin Frankel