You know that's not what I meant...

_________________
My blog about Macs and more: Kirkville

I see. The memory usage includes the OS' dynamic IO cache as well. What was confusing me was the part about applications holding on to memory but at the same time somehow being able to release it when someone else needed it. It's not actually the application that holds on to the memory but the OS. It's the presentation of used memory that makes it look like the memory is allocated by the application since it includes the OS' IO cache as well.

_________________
FusekiLibrary, an opening library.
SGF converter tools: Wbaduk NGF to SGF | 440 go problems | Fuseki made easy | Tesuji made easy | Elementary training & Dan level testing | Dan Tutor Shortcut To Dan

Yes, I think it's that apps can say they need to hold not memory, but also say that they don't need specific bit, so the OS can then reclaim it as needed. If you look at ram usage in OS X you can see that, at times, it will look as though it's all being used, and at others, a lot of it suddenly gets freed up. As best as I can understand, you really don't need to worry about ram unless you really don't have enough, or if you really use a lot.

_________________
My blog about Macs and more: Kirkville

Actually the applications are not aware of the IO caching. It's entirely up to the OS. If an application reads a file and there is enough free RAM, the OS caches it (or part of the file). The next time the application wants to read the same file it will get it from RAM instead which will be much faster. From the application's perspective it is just requesting to read a file. It does it in the same way both times. It's just that for the second read, the OS will decide to fetch it from RAM instead. If another program needs a lot of RAM the cached file will be thrown out from RAM and if it needs to be read again it will be read from disc.

_________________
FusekiLibrary, an opening library.
SGF converter tools: Wbaduk NGF to SGF | 440 go problems | Fuseki made easy | Tesuji made easy | Elementary training & Dan level testing | Dan Tutor Shortcut To Dan

An example par excellence of what it means to engage in reasonable and mature discourse. Well done, Carl!

_________________
"This is a game that rewards patience and balance. You must think like a man of action and act like a man of thought."
-Jonas Skarssen

Kirkmc, you seem to have your mind long made up about this issue, and you have ready-made excuses on why the opinion of those who disagree with you is marginal and not as valid as yours.

Why are you still trolling for responses here then? Why not just go ahead and write the article the way you clearly want to anyways?

Well, I was trying to point out that some people's perceptions were based on errors, and see if that made them change the way they thing. Ross's using a computer that doesn't meet the minimum system requirements, for example, yet saying that iTunes is bloated because it's slow on his computer. Or the memory issue, which is a very common comment, which really doesn't make that much of a difference. I was thinking that the more technically inclined people here might be willing to consider that some of their reasons could be mitigated by, well, reason. I guess I was wrong. (The only person interested in the memory question, admittedly quite technical, was CarlJung; I was hoping someone could tell me if Windows manages memory in a similar manner.)

No big deal; I've gotten plenty of comments from others who are more open minded on other forums.

_________________
My blog about Macs and more: Kirkville

I started to write an explanation of the memory issue, but decided not to post it, because it was too technical and not really important.

In general, you seem to display a layman's understanding of memory management, and you should not, perhaps, use that limited understanding to argue for or against memory bloat issues. It sometimes reads a bit like the famous Get Strong At Kibitz cover.

Windows and OSX (and other operating systems) manage memory in very similar ways. Swapping, for example, is not a choice by the program, but by the operating system. A program cannot "hold on" to physical memory. The OS decides which programs get to keep their stuff in RAM, and which get swapped to disk. A program can try to hold on to memory by using it, because the OS will generally swap out memory pages that are not used often, but that's no guarantee. And programs that use memory for the purpose of keeping it in RAM are frowned on, very much.

As to the bloat issue, I can say that when I was studying Computer Science, we were advised to write modules in such a way that they do one thing, and do it well. Don't tag on extra baggage. Don't write one module to do ten things, write ten modules that do one each. The same is often true for programs. Do one thing, do it well. The Mozilla project, for example, separated out the web browser and called it Firefox. Before that it was a web browser, email client, news reader, chat program, etc, etc, etc. Bloat. Firefox is much more efficient, it does one thing and does it well. But on the other hand, it was rather easy to have one program that allowed you to access everything on the internet in one go.

iTunes plays media, manages media, gives access to the store, synchronizes with your ipod, etc. If you don't have an ipod, then the ipod synch part is bloat. If you don't want to buy music online, then the store part is bloat. If you don't want to play music on your computer, then the music player is bloat. This is a choice that Apple made. They want thing to work out of the box. They want it so that if you buy music online, you don't have to go any extra steps to be able to play it, or put it on your iPod. You don't have to download a separate media player, you don't have to download a synch program. It just works. That doesn't mean it's not bloated, it just means there's a reason for the bloat: it's user friendly.

It's like buying a station wagon or SUV. You can buy a much smaller car and get around just as well. You can then rent a van when you need space to move stuff. But with the station wagon, you don't need to rent an extra van, it just works out of the box for transporting a lot of stuff. Yes, it is bloated, but that's a choice that you made. You sacrificed general efficiency in exchange for ease of use.

Personally, I prefer programs that do one thing well. But I'm computer savvy. I have no trouble at all finding and downloading applications that fit my needs. I have no objection to going trough some extra effort to get separate programs for every task. Most people just want things to work. The computer should be a magic box that just does what they want. That's why iTunes is the way it is.

I would guess that Windows has dynamic IO cache as well, but I'm not sure. Additionally it has something called SuperFetch that was introduced with Vista to make the computer snappier.

http://www.osnews.com/story/21471/Super ... orks_Myths

_________________
FusekiLibrary, an opening library.
SGF converter tools: Wbaduk NGF to SGF | 440 go problems | Fuseki made easy | Tesuji made easy | Elementary training & Dan level testing | Dan Tutor Shortcut To Dan

No, AFAIK Windows doesn't, at least not exactly. Using Windows you'll always have plenty of free memory, which surely seems a waste.
On Linux it's very similar to OSX, all free memory is used up as cache, and can be given back at any time.

But I think you're misunderstanding what this feature is actually doing. I'll try to elaborate:

First there is the distinction between virtual memory and physical memory. Each application has 4 GB of virtual memory space (on a 32 bit system, more on a 64 bit system). The application can request as much of that virtual memory as it wants, up to the 4 GB limit (actually 3 GB, because the operating system needs a bit of it). It doesn't really matter how much the application requests, as this is just virtual memory.

There are several uses for virtual memory:
- The application just needs some anonymous memory. When it requests the memory, nothing actually happens. On first use, physical memory is allocated. This memory stays allocated until the application explicitly frees it. This is the memory that actually matters.
- The application needs a file (e.g. a library): The file is mapped in the virtual memory. When it is first accessed, physical memory is allocated and a part of the file is read into it. This memory has the property that it can be freed at any time, at the cost of some performance because the file has to be read again on next use. This physical memory can also be shared between multiple processes that need the same file. This is especially important because there are many libraries in the system that are needed by most programs.
(there are a few more, but these are the most important ones)

Now let's get to the physical memory. Applications never use physical memory directly, it is managed by the operating system. The operating system can basically do the following with physical memory:
- Satisfy the virtual memory needs of applications.
- Use it as backing store for mapped files.
- Cache data from other file system operations.

What is really interesting for the performance of a system is of course the physical memory. Virtual memory basically never runs out (at least with 64 bit systems).
What happens now when physical memory runs out? The operating system has several possibilities:
- Swap some anonymous memory.
- Throw away the memory of a mapped file. It may be necessary to write changes to disk first.
- Reduce the file system cache size.

There is actually no way for the operating system to just "throw away" some memory of an application, it always has to be swapped. But it can throw away the backing store of a mapped file, since it can just re-load it from disk at any time. And of course it can do with the file system cache whatever it wants.
What will the operating system actually do? It depends on the situation. One might think that swapping is always worse as long as you can reduce the cache size, but that's not necessarily true. If memory is used relatively infrequent, swapping might well be the better choice.

The article you linked above does talk about total memory consumption, and why it is always near 100%. That's simply because the operating system is always filling it up with the file system cache, and that's nothing to worry about.

Memory consumption of single processes is a different matter. Here you basically have two numbers: Allocated memory, and committed memory (sometimes incorrectly called virtual and physical). Allocated memory is irrelevant, since that's just memory that the application requested but did not actually used. Committed memory on the other hand is memory the application actually uses, and the operating system can't do anything about it without sacrificing some performance. The only memory the operating system can just throw away without affecting anyone is the cache, but you will never see the cache counted towards the memory consumption of a process.

So if an application uses 300 MB memory according to the task manager, it actually needs it. It may be that some of this memory is only used infrequently, and there is not much harm in swapping it, but it still means the application needs the memory, at least as far as the operating system is concerned.

This explanation applies exactly to Linux and OSX (and probably every other Unix out there). It also mostly applies to Windows. It works slight different, but the conclusions are still the same.

But don't all modern programming tools allow such a modular approach within applications? (That's what other developers have told me.) The fact that features exist doesn't mean they're being used, or that they're using memory. They're called when needed, and therefore don't affect the program overall. It's not like a program needs to run all the code it has every time you click a button...

Regarding the memory thing, I'm not a developer, but I've discussed this with developers, and I certainly simplified. From what I understand, though, on OS X, a program will actually hold on to memory (or, to be more precise, store its code or data in memory pages) if it's using it, and after a certain time, it will free up that code or data allowing the OS to reclaim it. For this reason - and for years it's been like this - the standard advice given to people using Macs is that they basically never need to quit applications. As long as the app isn't doing anything, it'll get to a point when its memory can be reclaimed, and, in spite of what it may display as "real memory" that memory isn't being used. Naturally, swapping comes in to play here as well, but the memory figures that OS X displays are more theoretical than real.

_________________
My blog about Macs and more: Kirkville

That's not what I've been told regarding OS X. I've been told that "inactive memory" is part of what a task manager (or top) will show for a program as "real memory" (what you call "committed memory"). Because of this, the OS is much more flexible about allocating that memory to other apps. I can see it sometimes in a tool I have that monitors memory; there are some operations that can reduce the total amount of memory used by my Mac by a huge amount. This is not just system memory (what's called "wired memory" on OS X), but overall memory usage by app.

Also, on OS X, if a computer has more RAM, apps will use more memory. I have two Macs, one with 2 GB RAM and the other with 4 GB. The latter uses easily twice as much RAM for the same apps as the former, though the apps are doing the same thing. I think on OS X - again, from what I've been told by developers - has a kind of "monte carlo" system, like winning by a half-point, when it deals with memory. Rather than worrying about details, it just grabs a lot, then doles it out more efficiently.

_________________
My blog about Macs and more: Kirkville

"Inactive memory" is very similar to the file system cache, and is a part of the total memory summary ("system memory"). It is not counted towards processes, and it can indeed simply be considered as "free".

See http://support.apple.com/kb/ht1342



"Wired memory" is something entirely different. That's virtual memory that can not be swapped. It's mostly what the operating system itself (the kernel) uses. Applications usually should not request wired memory directly, but some do it anyway.



Yes, that's what I explained in my previous post
It's not the apps that will use more memory, it's the buffers and cache of the operating system.



Ok, that's a gross oversimplification. But to sum it up: Yes, the operating system does that. The applications do not.

(Sorry for not alternating quotes and texts...)

You say it's not that the apps use more memory, but the system shows that memory as being allocated to the apps. So what you say is more or less what I said, in a different way, that the figure shown as "real memory" is not a valid figure as to how much memory an app is using. Is that correct?

And the other point about the OS basically using what's available also can explain why, for one poster, iTunes is using 300 MB, or why, for me, it uses twice as much memory on one of my Macs as another.

My point being, as I tried to say above, that the amount of memory used by an app really means nothing. Though I'm not totally clear whether this number is closer to reality on Windows... From what you say above, this isn't the case; does that mean that any given app's memory figure is realistic, or is it higher or lower than the amount of memory that the app really _needs_?

_________________
My blog about Macs and more: Kirkville

Yes and no

Yes, all modern programming languages allow a modular approach, and modern programs can load modules when they are needed. There's also approaches like JIT compiling (Just-In-Time compiling), which allow programs to be more efficient by postponing compilation until needed, when more information is available to allow optimization of the compilation. Certainly, programs today are a lot more flexible in this regard. But loading modules takes time, and for many scenarios, developers do not consider that desirable. So they preload certain modules to make the application more responsive.

One of the most common complaints against Mozilla, back in the day, was that it took so long to start it. Internet Explorer loaded much much faster. That was because windows preloaded most of the modules required for IE. So the start-up was a matter of a few seconds, rather than the dozens of seconds that Mozilla took. So of course, people complained to Mozilla that their browser was slow. And so Mozilla introduced a "preloading" config option, where you could make the system load most of the Mozilla modules in advance, and make it keep them in memory when the browser closed, so that it started faster. Of course, those preloaded modules use memory. Even if you never used IE, those preloaded IE modules were there.

And programs that use a monolithic approach (one application does many things) will often cross-link their parts, because that gives a more pleasant user experience. If one part of the program is aware of the options provided by other parts, then it can do smart things. Context sensitive options. Previews. You like this song? There's more by this artist at the store! You've played this song a lot on your iPod, I'll put it in your favorites list, etc. So a few of those modules are already needed for the nice stuff.

Again, you're trading off a pleasant user experience with a certain amount of bloat. But if you do not have an iPod, and never buy music online, then its bloat that provides no benefit. It's mostly unobtrusive, but not quite. It's mostly costless, but not quite. So the people that complain about bloat the most are those that use the fewest options of the program.



flOvermind has written some excellent stuff on this issue, which covers most of this. Memory use is a tricky thing to measure. There's not just the IO cache, but also things like shared memory. The Chrome browser, for example, uses a separate process for each tab (and each plugin). Those processes share a lot of their memory. But a system tool may report each of them as using it. For an example of how this can mislead, see: http://www.ghacks.net/2009/06/21/web-br ... all-wrong/

Yes, the people who seem the most vehement about this, in my research this week, are those I queried in an audiobook forum. Most of them use iTunes only for loading audiobooks onto their iPods. Some use it for music, but relatively few. (As if, curiously, most people who are big audibooks fans really don't listen to music...) So they naturally see anything that does more than load audibooks as "bloat".

Which returns to the root question, though. Is a program bloated just because it has features that _you_ don't use? That's a bit self-centered. One person, in a classical music forum, said that the feature in iTunes that allows you to convert files to other formats is bloat. Another replied, no, this is a feature I use all the time. So in the end, there's no objective way to answer this question, and the whole point of my raising it was to see what subjective criteria people used to make this statement.

_________________
My blog about Macs and more: Kirkville

Although bloat is somewhat subjective, and the perception thereof depends on the user's needs, you can say that a program is more bloated in general if it has a feature set that is very disjunct and that almost none of its users need entirely. A feature set is disjunct if the features are mostly unrelated.

Most movie players, for example, can play tons of file formats and codecs. But that does not mean it is bloated, because all those codecs are closely related to the single purpose of the program: showing video. Photoshop has an enormous amount of features that most users never use. But all of them are related to the program's core function: editing images.

The old Netscape Communicator (and later Mozilla) on the other hand, includes web browsing, news reading and mail reading. Those are mostly unrelated features (their closest relation is probably that they are both "stuff that involves the internet"). That's where the bloat sets in.

iTunes, similarly, has a few functions that are somewhat disjunct. Playing music and putting files on iPod are related mostly as "stuff involving music". So it is more bloated, in general, than programs that play only music or only sync your device. As I said before, that's a choice Apple made. It helps the user, because it requires less effort on their part. It just works. But "It just works" comes at a price, because nothing is ever free. The price is a certain amount of bloat.

It is, I think, quite similar to the Windows vs. Linux issue. There is very little that Windows can do that Linux can't. The biggest difference, to the end user, is that on Windows stuff "just works". That's because Microsoft put in a ton of "just works" functionality at the cost of being more bloated. Linux is the OS of choice for a lot of technical stuff, because that's where technically knowledgeable people prefer to have a lean system that needs some expert setup, rather than a more bloated one that just works.

Which returns to the root problem: if anyone comments who does not use every single itunes feature and consider it essential, they are not the target user and are using it wrongly, and therefore are not qualified to comment.

Thus, itunes cannot possibly be bloated. Cunning.

No, I'm saying that the system does *not* show the cache as being allocated to the apps. It is only shown in the summary.



Buffers can't explain that, because they are not counted towards individual app memory usage. There must be some internal difference in the app. Perhaps one has a media library that's bigger the size? Who knows, I have no idea what's going on inside iTunes. I can only tell you what the memory management interface of the OS looks like.



Of course, the task manager will tell you just what the applications currently _uses_. Who knows whether it really _needs_ it?
But when the application _uses_ memory, other applications can't use it at the same time. If the app uses memory it doesn't actually need, that memory is occupied, too.

But on the other hand, I have to agree with you that the memory usage of an application should always be taken with a grain of salt. As programming guideline, I would certainly recommend that applications should just allocate memory and rely on the operating system to manage it, instead of trying to be smart. On the other hand, applications should also rely on the operating system cache and not do fancy caching themselves. Especially in the iTunes case, I'm pretty sure there is some internal caching with the media library going on, which would explain the vastly different memory usage numbers on different systems.

I'm not saying you're wrong in your conclusion. Actually I'm mostly agreeing with you (on the memory issue, I won't comment on the bloatedness discussion of iTunes, since I'm not an iTunes user). I'm just saying you didn't really understand it, giving plainly wrong arguments, which might be the reason why people don't believe it


Yes, and yes.

That's the definition of bloated. If a software contains lots of features that _I_ don't need and that get in _my_ way, _I_ consider a software bloated. That's entirely subjective and may vary from user to user. You can try to make it a bit more objective by making a statistic over the user base, that is, ask what percentage of its users consider the software bloated, and for what reasons. But most of the reasons for considering a software bloated will always be subjective.

Personally, I don't really use resource usage in my definition of bloated software. For me things as startup time are much more important. Also a very important factor for me is the user interface. For example, I consider MS Office (and also OpenOffice) bloated, because it has tons of buttons I never need, but that make it hard to find the features I want.

Of course that's self-centered. But I'm not using a software because I'm so nice to the developers of the software. I'm using the software because _I_ get an advantage from it. That's already self-centered in itself . If the software doesn't do what I want, or doesn't do it the way I want it, or uses too many resources and so on, then I won't use it anymore.

I think the problem with iTunes itself is not so much whether it's bloated or not, but that there is no alternative. If there were an alternative method to download files to the iPod, people wouldn't complain about iTunes. They would just switch to the alternative software. Or they wouldn't, but then you could tell everyone complaining they should switch. Either way, it wouldn't make much fuzz

With both iTunes and Windows - and, heck, Word, Photoshop or any other "big" program - I see it more as a trade-off between providing features and not requiring users to jump through hoops to put things together. I'm not sure I agree about the old Netscape, though; I remember using it, and I did see a clear relationship at the time among its different "modules". But that was then, in the early days of the Internet; it made a lot more sense. On the other hand, I've used Microsoft Entourage (their Mac email and everything else program), and that left me flummoxed. It combines a good e-mail program, with a mediocre calendar tool, and a very poor project manager. I never grokked why they added the project manager.

_________________
My blog about Macs and more: Kirkville