On 3/29/25 02:15, Kevin Stephenson wrote:
Bumping this old thread to clarify a few points.
As an initial note on terminology, a "server" can be called a server, an
instance, a server instance, or in PostgreSQL's (PG hereafter) case, a
"DB Cluster." They all are used interchangeably in the wild
e problems with that.
Perhaps the original poster would benefit from reviewing one of the gold
standards on this topic, "Designing Data-Intensive Applications: The Big Ideas
Behind Reliable, Scalable, and Maintainable Systems"
https://www.amazon.com/dp/1449373321
Thanks,
Kevin
__
Dear Laurenz
> I hear you, and I agree with that.
Thank you. Such a relief.
> If that worked well, then it should also work well with PostgreSQL and
foreign data wrappers.
You're right. We had problems with cross-server queries on SQL Server and
MariaDB too. It seems that cross-server queries a
On Thu, Mar 6, 2025 at 10:47 AM Igor Korot wrote:
> Hi,
>
> On Thu, Mar 6, 2025, 7:32 AM Greg Sabino Mullane
> wrote:
>
>> On Wed, Mar 5, 2025 at 9:44 PM me nefcanto wrote:
>>
>>> Anyway, that's why I asked you guys. However, encouraging me to go back
>>> to monolith without giving solutions on
Hi,
On Thu, Mar 6, 2025, 7:32 AM Greg Sabino Mullane wrote:
> On Wed, Mar 5, 2025 at 9:44 PM me nefcanto wrote:
>
>> Anyway, that's why I asked you guys. However, encouraging me to go back
>> to monolith without giving solutions on how to scale, is not helping.
>>
>
> We did. In addition to the
On 3/5/25 11:55, Laurenz Albe wrote:
On Wed, 2025-03-05 at 12:57 +0330, me nefcanto wrote:
Right now this data is in MariaDB, on separate databases (schema) but on one
server. The solution in this situation is to have a cross-database query.
(this is the status quo of our application)
Now our
On Wed, Mar 5, 2025 at 9:44 PM me nefcanto wrote:
> Anyway, that's why I asked you guys. However, encouraging me to go back to
> monolith without giving solutions on how to scale, is not helping.
>
We did. In addition to the ongoing FDW discussion, I mentioned read-only
replicas and Citus. As fa
On Thu, 2025-03-06 at 12:15 +0330, me nefcanto wrote:
> We had problems with cross-server queries on SQL Server and MariaDB too.
> It seems that cross-server queries are not solved by any engine. But we
> had no problem with cross-database queries. That's where it worked well
> both on SQL Server a
Hi,
On Thu, Mar 6, 2025, 1:44 AM me nefcanto wrote:
> I appreciate your time guys. Thank you very much.
>
> > Having 1 table per database per server is too ugly.
>
> Our databases are not one table per database. They are mapped to DDD's
> bounded contexts and usually by one table per domain enti
On Thu, 2025-03-06 at 06:13 +0330, me nefcanto wrote:
> I once worked with a monolithic SQL Server database with more than 10 billion
> records and about 8 Terabytes of data. A single backup took us more than 21
> days.
> It was a nightmare. Almost everybody knows that scaling up has a ceiling, bu
I appreciate your time guys. Thank you very much.
> Having 1 table per database per server is too ugly.
Our databases are not one table per database. They are mapped to DDD's
bounded contexts and usually by one table per domain entity.
For example, we have these databases:
- Contacts
- Co
On Wed, Mar 5, 2025 at 9:44 PM me nefcanto wrote:
> I once worked with a monolithic SQL Server database with more than 10
> billion records and about 8 Terabytes of data. A single backup took us more
> than 21 days. It was a nightmare.
>
25 years ago (meaning *much* slower hardware), I managed a
> On Mar 5, 2025, at 8:03 PM, Igor Korot jnit worked great for SQL Server. If
> you're small, we host them all on one server. If you get bigger, we can put
> heavy databases on separate machines.
>>
>> However, I don't have experience working with other types of database
>> scaling. I have us
Hi,
On Wed, Mar 5, 2025, 8:44 PM me nefcanto wrote:
> I once worked with a monolithic SQL Server database with more than 10
> billion records and about 8 Terabytes of data. A single backup took us more
> than 21 days. It was a nightmare. Almost everybody knows that scaling up
> has a ceiling, bu
I once worked with a monolithic SQL Server database with more than 10
billion records and about 8 Terabytes of data. A single backup took us more
than 21 days. It was a nightmare. Almost everybody knows that scaling up
has a ceiling, but scaling out has no boundaries.
Therefore I will never choose
On 3/5/25 04:15, me nefcanto wrote:
Dear Laurenz, the point is that I think if we put all databases into one
database, then we have blocked our growth in the future.
How?
A monolith database can be scaled only vertically. We have had huge
headaches in the past with SQL Server on Windows and a
On Wed, Mar 5, 2025 at 7:15 AM me nefcanto wrote:
> I think if we put all databases into one database, then we have blocked
> our growth in the future.
>
I think this is premature optimization. Your products table has 100,000
rows. That's very tiny for the year 2025. Try putting everything on on
On Wed, 2025-03-05 at 15:45 +0330, me nefcanto wrote:
> Dear Laurenz, the point is that I think if we put all databases into one
> database,
> then we have blocked our growth in the future.
Hard to say.
If you want to shard for horizontal scaling, that usually only works well
if there are few int
Dear Laurenz, the point is that I think if we put all databases into one
database, then we have blocked our growth in the future.
A monolith database can be scaled only vertically. We have had huge
headaches in the past with SQL Server on Windows and a single database.
But when you divide bounded c
On Wed, 2025-03-05 at 14:18 +0330, me nefcanto wrote:
> That means a solid monolith database. We lose many goodies with that.
> As a real-world example, right now we can import a single database
> from the production to the development to test and troubleshoot data.
Well, can't you import a single
Dear Laurenz,
That means a solid monolith database. We lose many goodies with that. As a
real-world example, right now we can import a single database from the
production to the development to test and troubleshoot data.
What if we host all databases on the same server and use FDW. What happens
i
On Wed, 2025-03-05 at 12:57 +0330, me nefcanto wrote:
> Right now this data is in MariaDB, on separate databases (schema) but on one
> server. The solution in this situation is to have a cross-database query.
> (this is the status quo of our application)
>
> Now our team has decided to migrate to
Laurenz Albe, thanks for your answer.
Right now this data is in MariaDB, on separate databases (schema) but on
one server. The solution in this situation is to have a cross-database
query. (this is the status quo of our application)
Now our team has decided to migrate to Postgres. However, we rea
On Wed, 2025-03-05 at 10:12 +0330, me nefcanto wrote:
> Adrian Klaver, thank you for the link. I asked the AI to create a query for
> me using FDW.
>
> The problem here is that it collects all of the product_id values from the
> ItemCategories table [...]
>
> That's not scalable. Is there a wor
Adrian Klaver, thank you for the link. I asked the AI to create a query for
me using FDW.
This is the sample query:
with filtered_products as (
select p.product_id
from products.product p
where p.title ilike '%search_term%'
), category_filtered as (
select ic.product_id
from t
On 3/4/25 20:40, me nefcanto wrote:
Hello
Consider this scenario:
* 3 servers, 3 databases, each on a separate server:
o *Products database*: Contains the *Products* table (with over
100,000 records).
o *Taxonomy database*: Contains the *Categories* and
*ItemCatego
Hello
Consider this scenario:
- 3 servers, 3 databases, each on a separate server:
- *Products database*: Contains the *Products* table (with over
100,000 records).
- *Taxonomy database*: Contains the *Categories* and *ItemCategories
(EAV)* tables.
- *Attributes d
27 matches
Mail list logo
