System engineer

[스크랩] SQL SERVER 스키마란?

99iberty — Sun, 7 Aug 2022 23:39:07 +0900

DB내 어떤 구조로 데이터가 저장되는가를 나타내는 데이터베이스 구조

https://truepia.tistory.com/167

[SQL2005] 스키마(Schema) 이해

SQL 서버 2005에서 데이터베이스에 테이블, 뷰, 저장프로시저와 같은 개체를 만들면 스키마(Schema)안에 만들어 집니다. SQL 서버 2000과 달라진 부분이기 때문에 SQL 서버 2000 사용자 분들이 이것에 대

truepia.tistory.com

SQL 서버 2005에서 데이터베이스에 테이블, 뷰, 저장프로시저와 같은 개체를 만들면 스키마(Schema)안에 만들어 집니다. SQL 서버 2000과 달라진 부분이기 때문에 SQL 서버 2000 사용자 분들이 이것에 대해 많이 혼란스러워합니다. 스키마에 대해 살펴보도록 하겠습니다.

1. 스키마란

스키마는 데이터베이스 개체에 대한 네임스페이스(Namespace) 입니다. 즉 데이터베이스 개체가 갖는 고유한 이름을 결정지어주게 됩니다. 스키마는 데이터베이스 안에서 고유해야 하기 때문에 데이터베이스의 모든 개체는 다음과 같은 형식의 고유한 이름을 갖게 됩니다.

Server.Database.Schema.Object

SQL 서버 2000에서는 Server.Database.Owner.Object와 같은 형식으로 이름이 결정되었습니다. 하지만 SQL 서버 2005에서는 개체의 소유자(Owner)가 아닌 스키마를 사용해 개체의 이름에서 소유자 부분을 분리시켰습니다. 이렇게 함으로써 얻어지는 이점이 있습니다.

1) 개체의 소유자와는 상관없이 스키마를 이용해 개체를 그룹화 하여 관리에 효율을 꾀할 수 있습니다.
2) 권한 관리의 경우 개체 개별 적으로 권한을 허용하지 않고 스키마에 권한을 허용하면 스키마에 포함된 모든 개체에 대해 자동으로 권한을 줄 수 있어 권한 관리가 효율적입니다.
3) 소유자 계정을 삭제하기 위해 소유자가 소유한 개테의 소유자를 변경하거나 개체를 삭제할 필요가 없습니다. SQL 서버 2000에서는 개체를 소유한 소유자의 계정을 삭제 할 수 없었습니다.

SQL 서버 2005의 예제 데이터베이스인 AdventureWorks 데이터베이스는 다음과 같은 스키마를 가지고 있습니다.

- HumanResources
- Person
- Production
- Purchasing
- Sales

Management Studio에서 AdventureWorks 데이터베이스의 테이블 부분을 확장하면 위 스키마로 시작된 테이블들을 확인 할 수 있습니다.

2. dbo 스키마

dbo 스키마는 모든 데이터베이스에 포함되어 있는 특수한 스키마입니다. 데이터베이스의 사용자 계정은 기본 스키마를 갖게 됩니다. 그리고 이 사용자가 테이블을 만들면 그 테이블은 사용자의 기본 스키마에 포함되게 됩니다. 예를 들어 기본 스키마가 Sales 인 사용자 계정이 Customer 테이블을 만들면 Sales.Customer와 같은 이름을 갖게 됩니다.

dbo 스키마는 별도의 기본 스키마가 지정되지 않은 사용자 계정의 기본 스키마가 됩니다. 만일 기본 스키마가 별도로 지정되지 않은 사용자 계정이 Orders 테이블을 만들면 기 테이블은 dbo.Orders 라는 이름을 갖게 됩니다.

3. 스키마 만들기

스키마는 SQL Server Management Studio 또는 CREATE SCHEMA 문을 이용해 만들 수 있습니다.

USE AdventureWorks
GO
CREATE SCHEMA Sales
GO

위 예제 에서는 AdventureWorks 데이터베이스에 Sales 스키마를 만들고 있습니다. 만일 AdventureWorks 데이터베이스에 Jane 이라는 사용자 계정이 있다고 가정한다면 다음과 같이 같이 Jane의 기본 스키마를 Sales로 지정 할 수 있습니다.

USE AdventureWorks
GO
ALTER USER Jane WITH DEFAULT_SCHEMA = Sales
GO

이러한 작업은 Management Studio에서도 가능합니다.

4. 스키마 삭제

스키마 삭제는 SQL Server Management Studio 또는 DROP SCHEMA 문을 이용하여 할 수 있습니다.

USE AdventureWorks
GO
DROP SCHEMA Sales
GO

5. 개체 이름 확인 과정

데이터베이스에 여러 개의 스키마가 있다면 개체 이름 확인은 단순하지 않습니다. 하지만 기본 스키마에 대해 정확히 파악하고 있다면 이 부분을 쉽게 이해 할 수 있습니다. 만일 데이터베이스에 다음과 같이 Person, Sales, dbo 스키마가 있다고 가정하겠습니다.

그리고 Person 스키마에는 Contact 테이블이 dbo 스키마에는 ErrorLog 테이블이 존재하고 있습니다. 물론 Sales 스키마에도 몇몇 테이블이 존재하고 있습니다. 이 상황에서 다음의 예에서 어떻게 개체를 찾게 되는지 확인을 해보도록 하겠습니다.

[예제1] 기본 스키마가 Person인 사용자 Lance가 다음 쿼리 문을 수행

SELECT * FROM Contact

개체 이름에 스키마가 생략되어 있으면 SQL 서버는 쿼리 문을 수행하는 사용자의 기본 스키마에서 해당 테이블을 찾게 됩니다. Lance의 기본 스키마가 Person이므로 위 쿼리 문 수행을 위해 SQL 서버는 Person.Contact 테이블을 찾게 됩니다. 그리고 Person.Contact 테이블에 존재하므로 결과를 리턴해 줍니다.

[예제2] 기본 스키마가 Sales 인 사용자 Anders가 다음 세 개의 쿼리 문을 수행

SELECT * FROM Person.Contact

Anders의 기본 스키마가 Sales 이지만 쿼리 문에 Person.Contact이라고 스키마가 지정되어 개체 이름을 사용했으므로 SQL 서버는 Person.Contact 테이블을 찾게 됩니다. 그리고 Person.Contact 테이블이 존재하므로 결과를 리턴해 줍니다.

SELECT * FROM ErrorLog

이 경우 개체 이름에 스키마가 생략되어 있으므로 SQL 서버는 쿼리 문을 수행하는 사용자의 기본 스키마에서 해당 테이블을 찾게 됩니다. Anders의 기본 스키마가 Sales이므로 Sales.ErrorLog 테이블을 찾게 됩니다. 하지만 Sales 스키마에는 ErrorLog 테이블이 없습니다. SQL 서버는 여기서 포기 하지 않고 dbo 스키마에 ErrorLog 테이블이 있는지 찾습니다. 즉 dbo.ErrorLog 테이블을 찾습니다. 그리고 dbo.ErrorLog 테이블이 있으므로 그 결과를 리턴해 줍니다.

SELECT * FROM Contact

역시 개체 이름에 스키마가 생략되어 있으므로 SQL 서버는 쿼리 문을 수행하는 사용자의 기본 스키마에서 해당 테이블을 찾게 됩니다. Anders의 기본 스키마가 Sales이므로 Sales.Contact 테이블을 찾습니다. 하지만 Sales 스키마에는 Contact 테이블이 없으므로 Sales.Contact 테이블 찾기는 실패합니다. SQL 서버는 이어서 dbo.Contact 테이블을 찾습니다. dbo 스키마에도 Contact 테이블이 없으므로 찾기에 실패합니다. 결국 이 쿼리 문은 개체를 찾지 못해 에러를 발생시킵니다.

정리 하면, 사용자의 기본 스키마에 속한 개체나 dbo 스키마에 속한 개체는 스키마를 생략해도 개체를 찾을 수 있으나, 그 외의 스키마에 속한 개체를 찾기 위해서는 개체 이름 앞에 스키마를 정확히 포함시켜주어야 한다는 것입니다. 가장 좋은 방법은 개체 이름 앞에 항상 스키마를 포함시키는 습관을 갖는 것입니다.

6. 정리

일반적으로 스키마라고 하면 테이블의 구조를 이야기 합니다. 하지만 SQL 서버 2005에서는 다른 의미를 가지고 있습니다. 스키마는 관리 및 보안을 위해 유익한 기능을 제공하고 있으며, SQL 서버 2005를 사용하려면 꼭 이해하고 있어야 하는 개체입니다. 본 강좌에서 다 다루지 못한 부분은 온라인 설명서를 통해 명확히 이해 하시는게 좋을것 같습니다.

[출처] 디비누리

출처: https://truepia.tistory.com/167 [진실세상을 꿈꾸며:티스토리]

https://smok95.tistory.com/151

[펌] SQL Server 사용자와 스키마 개념

SQL Server 2005 온라인 설명서(2007년 9월) 사용자와 스키마 분리 업데이트: 2007년 2월 1일 SQL Server 2005부터 각 개체는 데이터베이스 스키마에 속해 있습니다. 데이터베이스 스키마는 데이터베이스 사

smok95.tistory.com

https://docs.microsoft.com/ko-kr/previous-versions/sql/sql-server-2005/ms190387(v=sql.90)?redirectedfrom=MSDN

사용자와 스키마 분리

사용자와 스키마 분리 아티클 12/15/2008 읽는 데 12분 걸림 이 문서의 내용 --> 업데이트: 2006년 12월 12일 SQL Server 2005부터 각 개체는 데이터베이스 스키마에 속해 있습니다. 데이터베이스 스키마는

docs.microsoft.com

Understanding Logging and Recovery in SQL Server

99iberty — Mon, 18 Jul 2022 23:46:53 +0900

SQL Server: Understanding Logging and Recovery in SQL Server | Microsoft Docs

SQL Server: Understanding Logging and Recovery in SQL Server

Table of contents Article 10/04/2016 21 minutes to read In this article --> SQL Server Understanding Logging and Recovery in SQL Server Paul S. Randal At a Glance: How logging and recovery function in SQL Server How the transaction log works and what you n

docs.microsoft.com

At a Glance:

How logging and recovery function in SQL Server
How the transaction log works and what you need to know about managing it
Recovery models and their effect on logging

What Is Logging?
What Is Recovery?
The Transaction Log
Recovery Models
Wrapping Up

Some of the most misunderstood parts of SQL Server are its logging and recovery mechanisms. The fact that the transaction log exists and can cause problems if not managed correctly seems to confound many "involuntary DBAs." Why is it possible for the transaction log to grow unbounded? Why does it sometimes take so long for the database to come online after a system crash? Why can't logging be turned off completely? Why can't I recover my database properly? Just what is the transaction log and why is it there?

These are all questions I see repeatedly on SQL Server forums and newsgroups, so in this article I want to provide an overview of the logging and recovery system and explain why it is such an integral part of the SQL Server Storage Engine. I will explain the architecture of the transaction log and how the three recovery models available for a database can change the behavior of the transaction log and the logging process itself. Along the way, I'll also provide some links to resources covering transaction log management best practices.

What Is Logging?

Logging and recovery are not concepts that are unique to SQL Server—all commercial relational database management systems (RDBMSs) must have them to support the various ACID properties of transactions. ACID stands for Atomicity, Consistency, Isolation, and Durability, which are the fundamental properties of a transaction-processing system (such as an RDBMS). You can read more about this in the ACID Properties section of the MSDN Library.

Video (no longer available)

Paul Randal demonstrates the importance of managing your transaction log properly in the full recovery model, and he shows you techniques for doing so in SQL Server.

Operations in an RDBMS are logged (or recorded) at the physical and logical level in terms of what happens in the storage structures of the database. Each change to the storage structures has its own log record, which describes the structure being changed and what the change was. This is done in such a way that the change can be replayed or reversed, if necessary. The log records are stored in a special file called the transaction log—I will describe this in more detail later, but for now you can think of it as a sequential-access file.

A set of one or more such changes can be (and in fact, always are) grouped together in a transaction—which provides the basic unit of making changes (atomicity) to a database as far as users, application developers, and DBAs are concerned. A transaction either succeeds (committing) or fails/is canceled (rolling back). In the first case, the operations that form the transaction are guaranteed to be reflected in the database. In the second case, the operations are guaranteed not to be reflected in the database.

Transactions in SQL Server are either explicit or implicit. An explicit transaction is one where the user or application issues a BEGIN TRANSACTION T-SQL statement, signaling the start of a group of related changes by that session. An explicit transaction succeeds when a COMMIT TRANSACTION statement is issued, signaling the successful completion of the group of changes. If a ROLLBACK TRANSACTION statement is issued instead, all changes made by that session since the BEGIN TRANSACTION statement was issued are reverted (rolled back) and the transaction is aborted. A transaction rollback could also be forced by an external event, such as the database running out of disk space or a server crash, as I will explain later.

An implicit transaction is one where the user or application does not explicitly issue a BEGIN TRANSACTION statement before issuing a T-SQL statement. However, given that all changes to the database must be transactional, the Storage Engine will automatically start a transaction under the covers. When the T-SQL statement completes, the Storage Engine automatically commits the transaction that it started to wrap around the user's statement.

You might think that this is not necessary because a single T-SQL statement won't generate a large number of changes to the storage structures of the database, but consider something like an ALTER INDEX REBUILD statement. Although this statement can't be contained within an explicit transaction, it could generate an enormous number of changes to the database. So there must be some mechanism for ensuring that if something does go wrong (the statement is canceled, for instance), all of the changes are properly reversed.

As an example, consider what happens when a single table row is updated in an implicit transaction. Imagine a simple heap table with an integer column c1 and a char column c2. The table has 10,000 rows, and a user submits an update query as follows:

Copy

UPDATE SimpleTable SET c1 = 10 WHERE c2 LIKE '%Paul%';

The following operations take place:

The data pages from SimpleTable are read from disk into memory (the buffer pool) so they can be searched for matching rows. It turns out that three data pages hold five rows that match the WHERE clause predicate.
The Storage Engine automatically starts an implicit transaction.
The three data pages and five data rows are locked to allow the updates to occur.
The changes are made to the five data records on the three data pages in memory.
The changes are also recorded in log records in the transaction log on disk.
The Storage Engine automatically commits the implicit transaction.

Note that I didn't list a step where the three updated data pages are written back out to disk. This is because they don't yet need to be; as long as the log records describe the changes are on disk in the transaction log, then the changes are protected. If the pages need to be subsequently read or changed again, then the most up-to-date copy of the page is already in memory, just not on disk (yet). The data pages will be written out to disk when the next checkpoint operation occurs or if the memory that they use in the buffer pool is required for another page image.

Checkpoints exist for two reasons—to batch up write I/Os to improve performance and to reduce the time required for crash recovery. In terms of performance, if a data page were forced out to disk each time it was updated, the number of write I/Os occurring on a busy system could easily overwhelm the I/O subsystem. It's better to periodically write out dirty pages (pages that have been changed since being read from disk) than to write out pages immediately as they are changed. I'll discuss the recovery aspect of checkpoints in a moment.

One common misconception about checkpoints is that they only write out pages with changes from committed transactions. This is not true—a checkpoint always writes out all dirty pages, regardless of whether the transaction that changed a page has committed or not.

Write-ahead logging is the mechanism wherein the log records describing a change are written to disk before the changes themselves are written. It provides the durability part of the ACID properties. As long as the log records describing changes are on disk, in the event of a crash, the log records (and hence the changes themselves) can be recovered and the effects of the transaction are not lost.

What Is Recovery?

Looking for SQL Server Tips?

For tips on SQL Server, visit the TechNet Magazine SQL Server Tips page.

For more Tips on other products, visit the TechNet Magazine Tips index.

Logging exists to support a variety of operations in SQL Server. It ensures that if a crash occurs, a committed transaction will be correctly reflected in the database after the crash. It ensures that an uncommitted transaction will be correctly rolled back and not reflected in the database after a crash. It ensures that it is possible to cancel an in-flight transaction and have all its operations rolled back. It allows a backup copy of the transaction log to be taken so that a database can be restored and the transaction log backups replayed to bring the database to a specific point in time with transactional consistency. And it supports features that rely on reading the transaction log, such as replication, database mirroring, and change data capture.

The majority of these uses of logging involve a mechanism called recovery. Recovery is the process of having the changes described in log records replayed or reverted in the database. Replaying log records is called the REDO (or roll forward) phase of recovery. Reverting log records is called the UNDO (or roll back) phase of recovery. In other words, recovery will make sure that a transaction and all its constituent log records are either redone or undone.

The simple form of recovery is when a single transaction is canceled, in which case it is undone and there is no net effect on the database. The most complex form is crash recovery—when SQL Server crashes (for whatever reason) and the transaction log must be recovered to bring the database to a transactionally consistent point. This means that all transactions that were committed at the time of the crash must be rolled forward to ensure their effects are persisted in the database. And all in-flight transactions that had not committed at the time of the crash must be rolled back to ensure their effects are not persisted in the database.

This is because there is no facility for a transaction in SQL Server to continue after a crash. Thus, if the effects of a partially complete transaction were not rolled back, the database would be left in an inconsistent state (possibly even structurally corrupt, depending on what the transaction was in the middle of doing).

So how does recovery know what to do? All recovery processes depend on the fact that each log record is stamped with a log sequence number (LSN). A log sequence number is an ever-increasing, three-part number that uniquely defines the position of a log record within the transaction log. Each log record in a transaction is stored in sequential order within the transaction log and contains the transaction ID and the LSN of the previous log record for the transaction. In other words, each operation that is recorded as part of the transaction has a "link" back to the operation that immediately preceded it.

For the simple case of a single transaction being rolled back, the recovery mechanism can easily and quickly follow the chain of logged operations from the most recent operation back to the first operation and undo the effects of the operations in the opposite order from which they occurred. The database pages that were affected by the transaction are either still in the buffer pool or on disk. In either case, the image of the page that is available is guaranteed to be one where the effect of the transaction is reflected on the page and must be undone.

During crash recovery, the mechanism is more complicated. The fact that database pages are not written to disk when a transaction commits means that there is no guarantee that the set of database pages on disk accurately reflects the set of changes described in the transaction log—either for committed or uncommitted transactions. However, there is one final piece of the puzzle that I haven't mentioned yet—all database pages have a field in their page header (a 96-byte portion of the 8192-byte page that contains metadata about the page) that contains the LSN of the last log record that affected the page. This allows the recovery system to decide what to do about a particular log record that it must recover:

For a log record from a committed transaction where the database page has an LSN equal to or greater than the LSN of the log record, nothing needs to be done. The effect of the log record has already been persisted on the page on disk.
For a log record from a committed transaction where the database page has an LSN less than the LSN of the log record, the log record must be redone to ensure the transaction effects are persisted.
For a log record from an uncommitted transaction where the database page has an LSN equal to or greater than the LSN of the log record, the log record must be undone to ensure the transaction effects are not persisted.
For a log record from an uncommitted transaction where the database page has an LSN less than the LSN of the log record, nothing needs to be done. The effect of the log record was not persisted on the page on disk and as such does not need to be undone.

Crash recovery reads through the transaction log and ensures that all effects of all committed transactions are persisted in the database, and all effects of all uncommitted transactions are not persisted in the database—the REDO and UNDO phases, respectively. Once crash recovery completes, the database is transactionally consistent and available for use.

I mentioned earlier that one of the uses of a checkpoint operation is to reduce the amount of time that crash recovery takes. By periodically flushing out all dirty pages to disk, the number of pages that have changed because of committed transactions but whose images are not on disk is reduced. This, in turn, reduces the number of pages that need to have REDO recovery applied during crash recovery.

The Transaction Log

Crash recovery is only possible if the transaction log is intact. In fact, the transaction log is the most important part of the database—it's the only place where all changes to the database are guaranteed to be described in the event of a crash.

If the transaction log is missing or damaged after a crash, then crash recovery cannot complete, leading to a suspect database. In that case, the database must be restored from backups or recovered using less desirable options, such as emergency mode repair. (These procedures are beyond the scope of this article but will be covered in depth in articles later in the year.)

The transaction log is a special file that each database must have to function correctly. It holds the log records that are produced from logging and is used to read them again during recovery (or any of the other uses of logging I've already mentioned). As well as the space occupied by the log records themselves, a transaction will also reserve space in the transaction log for any potential log records required if the transaction were to be canceled and required to roll back. This accounts for the behavior you may observe where, say, a transaction that updates 50MB of data in the database may actually require 100MB of transaction log space.

When a new database is created, the transaction log is essentially empty. As transactions occur, log records are written sequentially to the transaction log, which means there is no performance gain from creating multiple transaction log files—a very common misconception. The transaction log will use each log file in turn.

Log records for concurrent transactions can be interspersed in the transaction log. Remember that log records for a single transaction are linked by their LSNs, so there is no need for all log records for a transaction to be grouped together in the log. LSNs can almost be thought of as a timestamp.

The physical architecture of the transaction log is shown in Figure 1. It is split internally into smaller chunks called virtual log files (or VLFs). These are simply an aid to easier internal management of the transaction log. When a VLF becomes full, logging automatically proceeds to use the next VLF in the transaction log. You might think that eventually the transaction log will run out of space, but this is where the transaction log is so different from data files.

Figure 1 Physical architecture of the transaction log

The transaction log is really a circular file—as long as the log records at the start of the transaction log have been truncated (or cleared). Then when logging reaches the end of the transaction log, it wraps around to the start again and begins overwriting what was there before.

So how do log records get truncated so the space they occupied can be reused? A log record is no longer needed in the transaction log if all of the following are true:

The transaction of which it is part has committed.
The database pages it changed have all been written to disk by a checkpoint.
The log record is not needed for a backup (full, differential, or log).
The log record is not needed for any feature that reads the log (such as database mirroring or replication).

A log record that is still needed is called active, and a VLF that has at least one active log record is also called active. Every so often, the transaction log is checked to see whether all the log records in a full VLF are active or not; if they are all inactive, the VLF is marked as truncated (meaning the VLF can be overwritten once the transaction log wraps). When a VLF is truncated, it is not overwritten or zeroed in any way—it is just marked as truncated and can then be reused.

This process is called log truncation—not to be confused with actually shrinking the size of the transaction log. Log truncation never changes the physical size of the transaction log—only which portions of the transaction log are active or not. Figure 2 shows the transaction log from Figure 1 after truncation has occurred.

Figure 2 The transaction log after log truncation

Active VLFs make up the logical log—the portion of the transaction log that contains all the active log records. The database itself knows where crash recovery should start reading log records within the active portion of the log—the start of the oldest active transaction in the log, the MinLSN (this is stored in the database boot page).

Crash recovery does not know where to stop reading log records, so it continues until it reaches a zeroed section of the transaction log (if the transaction log has not yet wrapped) or a log record whose parity bits do not fit the sequence from the previous log record.

As VLFs become truncated and new ones become active, the logical log moves within the physical transaction log file and eventually should wrap around to the start again, as shown in Figure 3.

Figure 3 The circular nature of the transaction log

The check whether log truncation can take place under either of the following circumstances:

When a checkpoint occurs in the SIMPLE recovery model or in other recovery models when a full backup has never been taken. (This implies that a database will remain in a pseudo-SIMPLE recovery model after being switched out of SIMPLE until a full database backup occurs.)
When a log backup completes.

Remember that log truncation may not be possible because of the many reasons a log record must remain active. When log truncation cannot occur, the VLFs cannot be truncated and eventually the transaction log has to grow (or another transaction log file be added). Excessive transaction log growth can cause performance problems through a phenomenon known as VLF fragmentation. Removing VLF fragmentation can sometimes lead to a dramatic improvement in the performance of log-related activities.

For more information on this, see Kimberly Tripp's blog post "8 Steps to Better Transaction Log Throughput." Kimberly discusses best practices pertaining to transaction log capacity planning, management, and performance improvements—it's well worth reading!

There are two common issues that can prevent log truncation:

A long-running active transaction. The entire transaction log since the first log record from the oldest active transaction can never be truncated until that transaction commits or aborts.
Switching to the FULL recovery model, taking a full backup, and then never taking any log backups. The entire transaction log will remain active, waiting to be backed up by a log backup.

For a complete list of factors and instructions on how to determine what is preventing log truncation, see the SQL Server Books Online topic "Factors that Can Delay Log Truncation." I've also created a video demonstration that shows the effect of uncontrolled transaction log growth and how to remove VLF fragmentation—check out this video screencast (and previous screencasts on SQL topics) at technetmagazine.com/videos.

If the transaction log does grow to capacity and cannot grow any further, then error 9002 will be reported and you will need to take steps to provide more space, such as growing the log file, adding another log file, or removing any impediment to the log being truncated.

Under no circumstances should you delete the transaction log, try to rebuild it using undocumented commands, or simply truncate it using the NO_LOG or TRUNCATE_ONLY options of BACKUP LOG (which have been removed in SQL Server 2008). These options will either cause transactional inconsistency (and more than likely corruption) or remove the possibility of being able to properly recover the database.

For more information on how to troubleshoot a full transaction log, check out the Books Online topic "Troubleshooting a Full Transaction Log (Error 9002)."

Recovery Models

As you can see, the behavior of the transaction log depends in part on the recovery model the database is using. There are three recovery models available, and they all have an effect on transaction log behavior or how operations are logged or both.

The FULL recovery model means that every part of every operation is logged, which is called being fully logged. Once a full database backup has been taken in the FULL recovery model, the transaction log will not automatically truncate until a log backup is taken. If you do not want to make use of log backups and the ability to recover a database to a specific point in time, do not use the FULL recovery model. However, if you wish to use database mirroring, then you have no choice, as it only supports the FULL recovery model.

The BULK_LOGGED recovery model has the same transaction log truncation semantics as the FULL recovery model but allows some operations to be partially logged, which is called being minimally logged. Examples are an index rebuild and some bulk-load operations—in the FULL recovery model the entire operation is logged.

But in the BULK_LOGGED recovery model only the allocation changes are logged, which drastically reduces the number of log records produced and, in turn, reduces the potential for transaction log growth. For more information on minimally logged operations, see the Books Online section "Operations that Can Be Minimally Logged."

Finally, the SIMPLE recovery model actually has the same logging behavior as the BULK_LOGGED recovery, but the transaction log truncation semantics are quite different. Log backups are not possible in the SIMPLE recovery model, which means the log can be truncated (as long as nothing else is holding log records active) when a checkpoint occurs. There are pros and cons to each of these recovery models in terms of which backups are possible (or needed) and the ability to recover to various points in time (I will cover this in another article later this year).

Wrapping Up

This article has really been a more academic explanation of how a critical part of SQL Server works. I hope I've managed to clear up any misconceptions you might have had. If this is your first introduction to logging and recovery, these are the key points I'd like you to take away from this article:

Do not create multiple log files, as it will not lead to a performance gain.
Be aware of the recovery model your database is using and the effect it has on the transaction log—especially around whether it can automatically truncate or not when a checkpoint occurs.
Be aware of the potential for transaction log growth, the factors that can lead to it, and how to get it back under control.
Know where to look for help when troubleshooting a full transaction log.

On my blog, I have a lot more information about the transaction log and factors that affect it—see my blog post category"Transaction Log" for more details. The various Books Online topics concerning the transaction log are also very good—starting with Transaction Log Management.

[스크랩] Lock 확인, 조치방법

99iberty — Mon, 18 Jul 2022 23:38:52 +0900

https://sagittariusof85s.tistory.com/124

[MS-SQL]락(LOCK)관리하기

DB를 사용하다보면 떨어질 수 없는 것이 LOCK 관리입니다. DB를 쓸 때 자주 발생하는 것이 LOCK이며, 최소한으로 발생되고 최대한 빠르게 처리가 되도록 해야 하는 것이 LOCK입니다. 그래서 이번에는

sagittariusof85s.tistory.com

DB를 사용하다보면

떨어질 수 없는 것이 LOCK 관리입니다.

DB를 쓸 때 자주 발생하는 것이

LOCK이며,

최소한으로 발생되고

최대한 빠르게 처리가 되도록 해야 하는 것이

LOCK입니다.

그래서 이번에는 LOCK 을

확인하고

조치하는 방법에 대해서 정리를 해보고자 합니다.

1. LOCK 확인하기

SQL SERVER에서는

락을 확인 하는 방법이

여러가지가 있습니다.

그 중 제가 아는 것은 3가지 방법이 있어서

그 3가지 방법에 대해

모두 정리를 해보고자 합니다.

1-1. SP_LOCK

MS-SQL에서 제공하는

시스템 프로시저가 있습니다.

이를 이용해서 확인할 수 있습니다.

함수는 SP_LOCK입니다.

사용방법은 다음과 같습니다.

EXEC SP_LOCK;

위처럼 작성하여

돌리면 됩니다.

그러면 결과는 다음과 같습니다.

그러면 Mode값이

X인 SPID가 있다면

해당 세션이 락 걸린 세션입니다.

1-2. SP_WHO2

또한, SQL SERVRVER에서 제공하는

또 다른 시스템프로시저가 있습니다.

SP_WHO2 입니다.

위 프로시저를 이용해서도

LOCK이 걸린 정보를 확인할 수 있습니다.

쿼리는 다음과 같습니다.

EXEC SP_WHO2;

위처럼 돌리면 결과는 다음과 같습니다.

위처럼 결과가 나옵니다.

이 곳에서 BlkBy 컬럼에 값이 있으면

Lock을 걸고 있는

프로세스의 spid값입니다.

1-3. SYS.sysprocesses

내부 시스템 테이블을 이용해서도

확인할 수 있습니다.

테이블은 SYS.sysprocesses 입니다.

쿼리는 다음과 같습니다.

SELECT * FROM SYS.sysprocesses WHERE blocked > 0;

위처럼 쿼리를 작성하여 돌리면 되십니다.

그러면 결과는 다음과 같습니다.

blocked에 값이 있으면

락이 걸린 것입니다.

저는 테스트서버이고,

그렇다보니 락 걸린 것이 없어서

안나오네요.

2. 쿼리 확인

해당 프로세스의 쿼리를

확인해 보는 것이 좋습니다.

정확하게 무슨 쿼리에서 락이 걸린 것인지 알 수 있고,

추후 튜닝을 통해

락이 안걸릴 수 있도록

조치할 수도 있으니

확인이 필요합니다.

확인 방법은 다음과 같습니다.

dbcc inputbuffer ( 75 );

위처럼 쿼리를 작성하여 돌리면 되십니다.

그러면 결과는 다음과 같습니다.

저는 테스트로

락 걸리지 않은 다른 프로세스의

spid값을 넣었습니다.

그래도 저렇게 쿼리가 나오는 것을 확인할 수 있습니다.

이렇게 쿼리 정보를 확인해놓는 것이

좋습니다.

3. kill

이제 종료를 시켜야 합니다.

종료 시키는 것은 매우 간단합니다.

방법은 다음과 같습니다.

kill SPID

위처럼 하면 됩니다.

그러면 해당 세션이 종료가 됩니다.

DB를 사용하면서

락은 떨어질래야 떨어질 수는 없습니다.

다만, 최소한으로 발생하도록 하여

DB의 성능을 높여주는 것이 중요합니다.

[스크랩] 오픈 트랜잭션 쿼리가 있는 DB 찾아보기

99iberty — Mon, 18 Jul 2022 23:37:00 +0900

https://www.overtop.co.kr/139

오픈 트랜잭션 쿼리가 있는 데이터베이스 찾아보기

현재 오픈된 트랜잭션이 어느 데이터베이스에 있는지 찾는 방법은 여러 가지 있다. 이 활성화된 세션을 찾아야 블로킹세션을 찾을 수 있다. 아래와 같이 세션 2개를 열어서 확인해보자 -- session #

www.overtop.co.kr

현재 오픈된 트랜잭션이 어느 데이터베이스에 있는지 찾는 방법은 여러 가지 있다.

이 활성화된 세션을 찾아야 블로킹세션을 찾을 수 있다.

아래와 같이 세션 2개를 열어서 확인해보자

-- session #1 tempdb 를 열고 실제 데이터베이스는 TD 에 있는 테이블을 업데이트 하고 있다.

use tempdb

begin tran

update A set diff = '1' from td.[dbo].[results] a

-- rollback

-- session #2

use tempdb

DBCC OPENTRAN

세션#1에서 tempdb를 오픈하고 Update를 하지만 실제는 TD 를 하고 있는 것을 찾아야 된다.

SELECT spid, dbid FROM SYS.SYSPROCESSES WHERE OPEN_TRAN = 1

select db_name(2)

현재 세션정보를 보면 tempdb 로 확인이 된다. 혹시나 오픈된 데이터베이스가 tempdb로 착각하면 안된다.

SELECT * FROM SYS.DM_TRAN_SESSION_TRANSACTIONS

위에서는 정말 잘못된 데이터베이스를 찾을수 있다. 실수하기 좋은 예이다.

실제 오픈 트랜잭션이 발생중인 데이터베이스를 찾으려면 sp_lock 으로 확인해보자.

sp_lock

select db_name(9) as DBName

DBName

use TD – 실제 오픈 트랜잭션이 있는 데이터베이스에서 DBCC OPENTRAN을 수행해야 결과를 볼 수 있다

DBCC OPENTRAN

SQL Server 실행 시간:

CPU 시간 = 0ms, 경과 시간 = 0ms

SQL Server 구문 분석 및 컴파일 시간:

CPU 시간 = 0ms, 경과 시간 = 0ms.

데이터베이스 'TD'의 트랜잭션 정보입니다.

가장 오래 전에 활성화된 트랜잭션:

SPID(서버 프로세스 ID): 54

UID(사용자 ID) : -1

이름 : user_transaction

LSN : (43:56:5)

시작 시간 : 09 14 2015 1:27:39:913PM

SID : 0x010500000000000515000000301322d263e67671bfb9b84833150000

DBCC 실행이 완료되었습니다. DBCC에서 오류 메시지를 출력하면 시스템 관리자에게 문의하십시오.

출처: https://www.overtop.co.kr/139 [AWSin:티스토리]

https://dallasdbas.com/dbcc-opentran-simplified/

Examples of how to use DBCC Opentran

Simple description of how to use DBCC Opentran

dallasdbas.com

n my Top 10 SQL Server Functions post awhile back, I listed DBCC OPENTRAN as one of the top 3, and for good reason.

An Open transaction may simply be something that has not finished yet, or someone issued a BEGIN TRAN without a corresponding COMMIT or ROLLBACK. Or as we will see at the end, replication is having issues.

You can use this against any database with minimal syntax and get back solid information very quickly.

--connect to sample db

use MyDatabase

--as generic as this command gets and still runs:

DBCC OPENTRAN

Result if nothing is open:

No active open transactions.
DBCC execution completed. If DBCC printed error messages, contact your system administrator.

If I start and execute a DML (insert, update or delete) transaction with BEGIN TRAN and leave out the corresponding COMMIT, I get:

Transaction information for database ‘SmallData_BigLog’.Oldest active transaction:
SPID (server process ID): 64 <———–
UID (user ID) : -1
Name : user_transaction
LSN : (637:4620:1)
Start time : Apr 1 2017 4:59:02:307PM
SID : 0x0105000000000005150000004893a0845595b6fef515cd5de9030000
DBCC execution completed. If DBCC printed error messages, contact your system administrator.

Now, if I open a second transaction (in a new query window) and execute any DML statement without the COMMIT, and then run DBCC OPENTRAN again, I get:

Yes…the same output, as this is just showing the ONE oldest transaction.

I can run a query to show that there are two SPIDs with open transactions:

SELECT spid, blocked,[dbid],last_batch,open_tran

FROM master.sys.sysprocesses

WHERE open_tran <> 0

If I COMMIT spid 64 and re-run DBCC OPENTRAN, the SPID changes to the second transaction I started:

Transaction information for database ‘SmallData_BigLog’.Oldest active transaction:
SPID (server process ID): 52 <———–
UID (user ID) : -1
Name : user_transaction
LSN : (637:9603:1)
Start time : Apr 1 2017 5:11:20:830PM
SID : 0x0105000000000005150000004893a0845595b6fef515cd5de9030000
DBCC execution completed. If DBCC printed error messages, contact your system administrator.

If I COMMIT spid 52 and re-run DBCC OPENTRAN along with checking sysprocesses for open_tran <> 0 I get:

No active open transactions.
DBCC execution completed. If DBCC printed error messages, contact your system administrator.spid blocked dbid last_batch open_tran
—— ——- —— ———————– ———(0 row(s) affected)

Now, all of that was just running DBCC OPENTRAN by itself. There are additional options:

1 2	--specify dbname, dbid or 0 for the current database DBCC OPENTRAN (SmallData_BigLog)

You will get results in the same format as the previous examples.

You can suppress all messages, regardless of if a transaction is open or not (but I have no idea why this would help you…)

1	DBCC OPENTRAN (0) with no_infomsgs

Result:

Command(s) completed successfully.

If you needed to periodically capture the oldest transaction, in order to review later, use WITH TABLERESULTS:

-- TableResults only shows the oldest open tran

-- useful running in a loop to load the oldest

-- tran over time.

--create a temp table

CREATE TABLE #OpenTranStatus (

ActiveTransaction varchar(25),

Details sql_variant

);

-- Execute the command, putting the results in the table.

INSERT INTO #OpenTranStatus

EXEC ('DBCC OPENTRAN (SmallData_BigLog) with tableresults')

SELECT * FROM #OpenTranStatus

DROP TABLE #OpenTranStatus

In the above, you could create a user table instead of a temp table of course…it depends on your needs.

One more particularly useful item you may see when running DBCC OPENTRAN by itself:

Transaction information for database ‘Music’.Replicated Transaction Information:
Oldest distributed LSN : (37:143:3)
Oldest non-distributed LSN : (37:144:1)
DBCC execution completed. If DBCC printed error messages, contact your system administrator.

If your database is participating in Replication as a Publisher, this may show up when running OPENTRAN, but it doesn’t necessarily mean that the transaction is actually open. I set this up and stopped the Replication Log Reader and Distribution agent jobs. I then added some data to a published table (article) and ran DBCC OPENTRAN to get the above result. Note that there are two lines with LSN information in them (no SPIDs)

I then ran the Log Reader Agent job and got back:

Transaction information for database ‘Music’.Replicated Transaction Information:
Oldest distributed LSN : (37:157:3)
Oldest non-distributed LSN : (0:0:0)
DBCC execution completed. If DBCC printed error messages, contact your system administrator.

I have verified that new records I inserted have been read by the log reader, AND distributed to the subscriber(s). This means that while you are seeing

Oldest distributed LSN : (37:157:3)

There is not an error…just info.

If you have non-distributed LSNs, there is something to troubleshoot in the replication process which is way outside the scope of this post. A non-distributed replicated transaction/LSN CAN cause some huge Log file growth, and need to be investigated. If this happens frequently, use the TABLERESULTS option to log to a regular table and alert on it.

Hopefully this gives you some insight into various ways to use DBCC OPENTRAN as well as use cases for the various options. 90% of the time I run this, it is due to application transactions timing out, or log file growth issues.

[스크랩] Lazy Log Truncation - Clearing of SQL transaction log VLF status deferred

99iberty — Mon, 18 Jul 2022 23:36:13 +0900

Lazy Log Truncation - Clearing of SQL transaction log VLF status deferred - Microsoft Tech Community

Lazy Log Truncation - Clearing of SQL transaction log VLF status deferred

First published on MSDN on Jun 27, 2013 I came across an oddity in the transaction log when I was testing Availability Group backup scenarios which really had me confused. It required some help from my colleague Denzil and a few others to figure out what

techcommunity.microsoft.com

First published on MSDN on Jun 27, 2013

I came across an oddity in the transaction log when I was testing Availability Group backup scenarios which really had me confused. It required some help from my colleague Denzil and a few others to figure out what was going on. This adventure taught us about Lazy Log Truncation. This is not new to SQL 2012 as it also exists in SQL 2008R2 (and possibly prior), but only when a database is being mirrored or participating in an Availability Group.

For more on VLF’s and transaction log architecture: Transaction Log Physical Architecture

Before getting into the oddity that Lazy Log Truncation brings into play, let’s first look at what you normally see in the transaction log as it is filled and truncated.

I started off creating a database with a fixed log file size of 100MB in full recovery model. I have this all in the attached script for you to try on your own as well. Once that is set up, take a full database backup to get full recovery model going. Then run DBCC LOGINFO to show you the current VLFs in the database.

USE [LLTTest]
GO
DBCC LOGINFO --you see here that there are 8 VLFs with 1 that has a status of 2
GO

The status value of 2 means that it is the active portion of the log, meaning that it contains data that needs to be backed up in order for that VLF to be able to be used again.

Next we will do some work in the database that will generate a little log activity.

CREATE TABLE test1 (myvarchar VARCHAR(5000))
GO
INSERT INTO test1
VALUES (replicate('*', 5000)) GO 5000
DELETE
FROM test1

Now take a look at DBCC LOFINGO again and you can see that more of the VLFs contain active transaction log data.

Next, we take a backup of the transaction log in order to clear out all the full VLFs and run DBCC LOGINFO again. Note that FSeqNo 35 is still marked as active. This is because only full VLFs that do not contain log records for active transactions or records needed for replication (mirroring, CDC, replication, etc) are truncated. Also note that there must always be at least 1 active VLF.

BACKUP LOG [LLTTest] TO DISK = N'C:\Program Files\Microsoft SQL Server\MSSQL11.MSSQLSERVER\MSSQL\Backup\LLTTest.bak'
WITH NOFORMAT
	,NOINIT
	,NAME = N'LLTTest-Transaction Log Backup'
	,SKIP
	,NOREWIND
	,NOUNLOAD
	,STATS = 10
GO
DBCC LOGINFO --after the tlog backup all but one of the VLFs are inactive (status 0)
GO

This is exactly what you would expect to see, no surprises here. When it becomes interesting is when you do this same test against a database participating in an Availability Group. For the sake of keeping this blog entry from being too long, I am not showing you how to add this db to an availability group, but you can do so on your own using the GUI tools.

After adding this database to an availability group, there was no change to the output of DBCC LOGINFO, but one interesting thing to note is that the DBCC LOGINFO output will be identical across all the replicas when everything is synchronized.

Now, we are doing to do the same work we did before to put some activity in the transaction log.

INSERT INTO test1
VALUES (replicate('*', 5000)) GO 5000
GO
DELETE
FROM test1
GO
DBCC LOGINFO --Nothing special here, you can see more active VLFs
GO

This looks similar to what we saw before, it is just that different VLFs are being used and the activity rolled over to the beginning of the log file since I have the file set to a fixed size (to make this easier to demo).

Now we will backup the transaction log to truncate the VLFs, right?

BACKUP LOG [LLTTest] TO DISK = N'C:\Program Files\Microsoft SQL Server\MSSQL11.MSSQLSERVER\MSSQL\Backup\LLTTest.bak'
WITH NOFORMAT
	,NOINIT
	,NAME = N'LLTTest-Transaction Log Backup'
	,SKIP
	,NOREWIND
	,NOUNLOAD
	,STATS = 10
GO
DBCC LOGINFO
GO

ACK! The status didn't clear to 0 on any of the VLFs?! The DBCC LOGINFO output did not change at all. The replica is fully synchronized, so we know it is not due to data not being at all the replicas. The transaction log backup file does contain data, so are those VLFs cleared out or not?

If we do work again, what will happen? We think the transaction log was just backed up so there should be plenty of free space right? Running DBCC SQLPERF ('LOGSPACE') confirms that. So what will happen? Will the transaction log fill and return an error? This seems very confusing. Well let's give it a shot!

INSERT INTO test1
VALUES (replicate('*', 5000)) GO 5000
DELETE
FROM test1
	--Spoiler Alert:  IT WORKS!!
GO
DBCC LOGINFO
GO

You can see that FSeqNo 32-35 have been replaced with 40-43. Even though the VLFs were still considered active, they were able to be reused since they were part of the previous transaction log backup.

We noticed that in a mirroring or Availability Group scenario, Lazy Log Truncation marks the VLFs for truncation, but it does not actually truncate the log unless those VLFs are necessary. Why is this done? Well, the primary log truncation has to retain log which may be in use at any of the secondaries. When adding a replica, this aids in minimizing the number of transaction log backups that must be restored in order to get the replica transaction log at the same point as the primary.

Is this something that you need to be aware of? Not really, but we thought it was an interesting tidbit about how Availability Groups and mirroring have had so much thought put into them. We also didn’t want you to be concerned if you happened to notice that a lot of your VLFs are marked as active when you think they have been backed up and should be truncated.

Lisa Gardner – Premier Field Engineer

Linux VM root로 로그인하기

99iberty — Tue, 14 Dec 2021 23:29:39 +0900

$ sudo su -

# vim /etc/ssh/sshd_config

#LoginGraceTime 2m
#PermitRootLogin prohibit-password
PermitRootLogin yes
#StrictModes yes
#MaxAuthTries 6
#MaxSessions 10

저장

# systemctl restart sshd

[스크랩] Understanding MariaDB Architecture

99iberty — Thu, 4 Nov 2021 23:22:52 +0900

https://mariadb.com/kb/en/understanding-mariadb-architecture/

Understanding MariaDB Architecture

An overview of MariaDB server architecture, especially where it differs from SQL Server.

mariadb.com

Understanding MariaDB Architecture

MariaDB architecture is partly different from the architecture of traditional DBMSs, like SQL Server. Here we will examine the main components that a new MariaDB DBA needs to know. We will also discuss a bit of history, because this may help understand MariaDB philosophy and certain design choices.

This section is an overview of the most important components. More information is included in specific sections of this migration guide, or in other pages of the MariaDB Knowledge Base (see the links scattered over the text).

Storage Engines

MariaDB was born from the source code of MySQL, in 2008. Therefore, its history begins with MySQL.

MySQL was born at the beginning of the 90s. Back in the days, if compared to its existing competitors, MySQL was lightweight, simple to install, easy to learn. While it had a very limited set of features, it was also fast in certain common operations. And it was open source. These characteristics made it suitable to back the simple websites that existed at that time.

The web evolved rapidly, and the same happened to MySQL. Being open source helped a lot in this respect, because the community needed functionalities that weren’t supported at that time.

MySQL was probably the first database system to support a pluggable storage engine architecture. Basically, this means that MySQL knows very little about creating or populating a table, reading from it, building proper indexes and caches. It just delegated all these operations to a special plugin type called a storage engine.

One of the first plugins developed by third parties was InnoDB. It is very fast, and it adds two important features that are not otherwise supported: transactions and foreign keys.

Note that when MariaDB asks a storage engine to write or read a row, the storage engine could theoretically do anything. This led to the creation of very interesting alternative engines, like BLACKHOLE (which doesn’t write or read any data, acting like the /dev/null file in Linux), or CONNECT (which can read and write to files written in many different formats, or remote DBMSs, or some other special data sources).

Nowadays InnoDB is the default MariaDB storage engine, and it is the best choice for most use cases. But for particular needs, sometimes using a different storage engine is desirable. In case of doubts about the best storage engine to use for a specific case, check the Choosing the Right Storage Engine page.

When we create a table, we specify its storage engine or use the default one. It is possible to convert an existing table to another storage engine, though this is a blocking operation which requires a complete table copy. Third-party storage engines can also be installed while MariaDB is running.

Note that it is perfectly possible to use tables with different storage engines in the same transaction (even if some engines are not transactional). It is even possible to use different engines in the same query, for example with JOINs and subqueries.

The default storage engine can be changed by changing the default_storage_engine variable. A different default can be specified for temporary tables by setting default_tmp_storage_engine. MariaDB uses Aria for system tables and temporary tables created internally to store the intermediate results of a query.

InnoDB

It is worth spending some more words here about InnoDB, the default storage engine.

Primary Key and Indexes

InnoDB primary keys are always the equivalent of SQL Server clustered indexes. In other words, an InnoDB table is always ordered by the primary key.

If an InnoDB table doesn't have a user-defined primary key, the first UNIQUE index whose columns are all NOT NULL is used as a primary key. If there is no such index, the table will have a clustered index. The terminology here can be a bit confusing for SQL Server and other DBMS users. A clustered index in InnoDB is a 6 bytes value that is added to the table. This index and its values are completely invisible to the users. It's important to note that clustered indexes are governed by a global mutex that greatly reduces their scalability.

Secondary indexes are ordered by the columns that are part of the index, and contain a reference to each entry's corresponding primary key value.

Some consequences of these design choices are the following:

For performance reasons, a primary key value should be inserted in order. In other words, the last inserted value should be the highest. This order is normally followed when inserting values into an AUTO_INCREMENT primary key. The reason is that inserting values in the middle of an ordered data structure is slower, unless they fit into existing holes. If we insert primary key values randomly, InnoDB often has to rearrange pages to make some room for the new data.
A big primary keys means that all secondary indexes are also big.
A query by primary key will require a single search. A query on a secondary index that also reads columns not contained in the index will require one search on the index, plus one more search for each row that satisfies the index condition.
We shouldn't explicitly include the primary key in a secondary index. If we do so, the primary key column will be duplicated in the index.

Tablespaces

For InnoDB, a tablespace is a file containing data (not a file group as in SQL Server). The types of tablespaces are:

The system tablespace is stored in the file ibdata. It contains information used by InnoDB internally, like rollback segments, as well as some system tables. Historically, the system tablespace also contained all tables created by the user. In modern MariaDB versions, a table is created in the system tablespace only if the innodb_file_per_table system variable is set to 0 at the moment of the table creation. By default, innodb_file_per_table is 1.

Tables created while innodb_file_per_table=1 are written into their own tablespace. These are .ibd files.

Starting from MariaDB 10.2, temporary tables are written into temporary tablespaces, which means ibtmp* files. Previously, they were created in the system tablespace or in file-per-table tablespaces according to the value of innodb_file_per_table, just like regular tables. Temporary tablespaces, if present, are deleted when MariaDB starts.

It is important to remember that tablespaces can never shrink. If a file-per-table tablespace grows too much, deleting data won't recover space. Instead, a new table must be created and data needs to be copied. Finally, the old table will be deleted. If the system tablespace grows too much, the only solution is to move data into a new MariaDB installation.

Transaction Logs

In SQL Server, the transaction log contains both the undo log and the redo log. Usually we have only one transaction log.

In MariaDB the undo log and the redo log are stored separately. By default, the redo log is written to two files, called ib_logfile0 and ib_logfile1. The undo log by default is written to the system tablespace, which is in the ibdata1 file. However, it is possible to write it in separate files in a specified directory.

MariaDB provides no way to inspect the contents of the transaction logs. However, it is possible to inspect the binary log.

InnoDB transaction logs are written in a circular fashion: their size is normally fixed, and when the end is reached, InnoDB continues to write from the beginning. However, if very long transactions are running, InnoDB cannot overwrite the oldest data, so it has to expand the log size instead.

InnoDB Buffer Pool

MariaDB doesn't have a central buffer pool. Each storage engine may or may not have a buffer pool. The InnoDB buffer pool is typically assigned a big amount of memory. See MariaDB Memory Allocation.

MariaDB has no extension like the SQL Server buffer pool extension.

A part of the buffer pool is called the change buffer. It contains dirty pages that have been modified in memory and not yet flushed.

InnoDB Background Threads

InnoDB has background threads that take care of flushing dirty pages from the change buffer to the tablespaces. They don't directly affect the latency of queries, but they are very important for performance.

SHOW ENGINE InnoDB STATUS shows information about them in the BACKGROUND THREAD section. They can also be seen using the threads table, in the performance_schema.

InnoDB flushing is similar to lazy writes and checkpoints in SQL Server. It has no equivalent for eager writing.

For more information, see InnoDB Page Flushing and InnoDB Purge.

Checksums and Doublewrite Buffer

InnoDB pages have checksums. After writing pages to disk, InnoDB verifies that the checksums match. The checksum algorithm is determined by innodb_checksum_algorithm. Check the variable documentation for its consequences on performance, backward compatibility and encryption.

In case of a system crash, hardware failure or power outage, a page could be half-written on disk. For some pages, this causes a disaster. Therefore, InnoDB writes essential pages to disk twice. A backup copy of the new page version is written first. Then, the old page is overwritten. The backup copies are written into a file called the doublewrite buffer.

If an event prevents the first page from being written, the old version of the page will still be available.
If an event prevents the old page from being completely overwritten by its new version, the page can still be recovered using the doublewrite buffer.

The doublewrite buffer can disabled using the innodb_doublewrite variable, but this usually doesn't bring big performance benefits. The doublewrite buffer location can be changed with innodb_doublewrite_file.

Aria

Even if we only create InnoDB tables, we use Aria indirectly, in two ways:

For system tables.
For internal temporary tables.

Aria is a non-transactional storage engine. By default it is crash-safe, meaning that all changes to data are written and fsynced to a write-ahead log and can always be recovered in case of a crash.

Aria caches indexes into the pagecache. Data are not directly cached by Aria, so it's important that the underlying filesystem caches reads and writes.

The pagecache size is determined by the aria_pagecache_buffer_size system variable. To know if it is big enough we can check the proportion of free pages (the ratio between Aria_pagecache_blocks_used and Aria_pagecache_blocks_unused) and the proportion of cache misses (the ratio between Aria_pagecache_read_requests and Aria_pagecache_reads.

The proportion of dirty pages is the ratio between Aria_pagecache_blocks_used and Aria_pagecache_blocks_not_flushed tells us if the log file is big enough.

The size of Aria log is determined by aria_log_file_size.

Databases

MariaDB does not support the concept of schema. In MariaDB SQL, schema and schemas are synonyms for database and databases.

When a user connects to MariaDB, they don't connect to a specific database. Instead, they can access any table they have permissions for. There is however a concept of default database, see below.

A database is a container for database objects like tables and views. A database serves the following purposes:

A database is a namespace.
A database is a logical container to separate objects.
A database has a default character set and collation, which are inherited by their tables.
Permissions can be assigned on a whole database, to make permission maintenance simpler.
Physical data files are stored in a directory which has the same name as the database to which they belong.

System Databases

MariaDB has the following system databases:

mysql is for internal use only, and should not be read or written directly.
information_schema contains all information that can be found in SQL Server's information_schema and more. However, while SQL Server's information_schema is a schema containing information about the local database, MariaDB's information_schema is a database that contains information about all databases.
performance_schema contains information about MariaDB runtime. It is disabled by default. Enabling it requires setting the performance_schema system variable to 1 and restarting MariaDB.

Default Database

When a user connects to MariaDB, they can optionally specify a default database. A default database can also be specified or changed later, with the USE command.

Having a default database specified allows one to specify tables without specifying the name of the database where they are located. If no default database is specified, all table names must be fully qualified.

For example, the two following snippets are equivalent:

SELECT * FROM my_database.my_table; -- is equivalent to: USE my_database; SELECT * FROM my_table;

Even if a default database is specified, tables from other databases can be accessed by specifying their fully qualified names:

-- this query joins my_database.my_table to your_database.your_table USE my_database; SELECT m.* FROM my_table m JOIN your_database.your_table y ON m.xyz = y.xyz;

MariaDB has the DATABASE() function to determine the current database:

SELECT DATABASE();

Stored procedures and triggers don't inherit a default database from the session, nor by a caller procedure. In that context, the default database is the database which contains the procedure. USE can be used to change it. The default database will only be valid for the rest of the procedure.

The Binary Log

Different tables can be built using different storage engines. It is important to note that not all engines are transactional, and that different engines implement the transaction logs in different ways. For this reason, MariaDB cannot replicate data from a master to a slave using an equivalent of SQL Server transactional replication.

Instead, it needs a global mechanism to log the changes that are applied to data. This mechanism is the binary log, often abbreviated to binlog.

The binary log can be written in the following formats:

STATEMENT logs SQL statements that modify data;
ROW logs a reference to the rows that have been modified, if any (usually it’s the primary key), and the new values that have been added or modified, in a binary format.
MIXED is a combination of the above formats. It means that ROW is used for statements that can safely be logged in this way (see below), and STATEMENT is used in other cases. This is the default format from MariaDB 10.2.

In most cases, STATEMENT is slower because the SQL statement needs to be re-executed by the slave, and because certain statements may produce a different result in the slave (think about queries that use LIMIT without ORDER BY, or the CURRENT_TIMESTAMP() function). But there are exceptions, and besides, DDL statements are always logged as STATEMENT to avoid flooding the binary log. Therefore, the binary log may well contain both ROW and STATEMENT entries.

See Binary Log Formats.

The binary log allows:

replication, if enabled on the master;
promoting a slave to a master, if enabled on that slave;
incremental backups;
seeing data as they were in a point of time in the past (flashback);
restoring a backup and re-appling the binary log, with the exception of a data change which caused problems (human mistake, application bug, SQL injection);
Capture Data Changes (CDC), by streaming the binary log to technologies like Apache Kafka.

If you don't plan to use any of these features on a server, it is possible to disable the binary log to slightly improve the performance.

The binary log can be inspected using the mysqlbinlog utility, which comes with MariaDB. Enabling or disabling the binary log requires restarting MariaDB.

See also MariaDB Replication Overview for SQL Server Users and MariaDB Backups Overview for SQL Server Users for a better understanding of how the binary log is used.

Plugins

Storage engines are a special type of plugin. But others exist. For example, plugins can add authentication methods, new features, SQL syntax, functions, informative tables, and more.

A plugin may add some server variables and some status variables. Server variables can be used to configure the plugin, and status variables can be used to monitor its activities and status. These variables generally use the plugin's name as a prefix. For example InnoDB has a server variable called innodb_buffer_pool_size to configure the size of its buffer pool, and a status variable called Innodb_pages_read which indicates the number of memory pages read from the buffer pool. The category system variables of the MariaDB Knowledge Base has specific pages for system and status variables associated with various plugins.

Many plugins are installed by default, or available but not installed by default. They can be installed or uninstalled at runtime with SQL statements, like INSTALL PLUGIN, UNINSTALL PLUGIN and others; see Plugin SQL Statements. 3rd party plugins can be made available for installation by simply copying them to the plugin_dir.

It is important to note that different plugins may have different maturity levels. It is possible to prevent the installation of plugins we don’t consider production-ready by setting the plugin_maturity system variable. For plugins that are distributed with MariaDB, the maturity level is determined by the MariaDB team based on the bugs reported and fixed.

Some plugins are developed by 3rd parties. Even some 3rd party plugins are included in MariaDB official distributions - the ones available on mariadb.org.

In MariaDB every authorization method (including the default one) is provided by an authentication plugin. A user can be required to use a certain authentication plugin. This gives us much flexibility and control. Windows users may be interested in gsapi (which supports Windows authentication, Kerberos and NTLM) and named_pipe (which uses named pipe impersonation).

Other plugins that can be very useful include userstat, which includes statistics about resources and table usage, and METADATA_LOCK_INFO, which provides information about metadata locks.

Thread Pool

MariaDB supports thread pool. It works differently on UNIX and on Windows. On Windows, it is enabled by default and its implementation is quite similar to SQL Server. It uses the Windows native CreateThreadpool API.

If we don't use the thread pool, MariaDB will use its traditional method to handle connections. It consists of using a dedicated thread for each client connection. Creating a new thread has a cost in terms of CPU time. To mitigate this cost, after a client disconnects, the thread may be preserved for a certain time in the thread cache.

Whichever connection method we use, MariaDB has a maximum number of simultaneous connections, which can be changed at runtime. When the limit is reached, if more clients try to connect they will receive an error. This prevents MariaDB from consuming all the server resources and freezing or crashing. See Handling Too Many Connections.

Configuration

MariaDB has many settings that control the server behavior. These can be set up when starting mysqld (mysqld options), and the vast majority are also accessible as server system variables. These can be classified in these ways:

Dynamic or static;
Global, session, or both.

Note that server system variables are not to be confused with user-defined variables. The latter are not used for MariaDB configuration.

Configuration Files

MariaDB can use several configuration files. Configuration files are searched in several locations, including in the user directory, and if present they all are read and used. They are read in a consistent order. These locations depend on the operating system; see Default Option File Locations. It is possible to tell MariaDB which files it should read; see Global Options Related to Option Files.

On Linux, by default the configuration files are called my.cnf. On Windows, by default the configuration files can be called my.ini or my.cnf. The former is more common.

If a variable is mentioned multiple times in different files, the occurrence that is read last will overwrite the others. Similarly, if a variable is mentioned several times in a single file, the occurrence that is read last overwrites the others.

The contents of each configuration file are organized by option groups. MariaDB Server and client programs read different groups. The read groups also depend on the MariaDB version. See Option Groups for the details. Most commonly, the [server] or [mysqld] groups are used to contain all server configuration. The [client-server] group can be used for options that are shared by the server and the clients (like the port to use), to avoid repeating those variables multiple times.

Dynamic and Static Variables

Dynamic variables have a value that can be changed at runtime, using the SET SQL statement. Static variables have a value that is decided at startup (see below) and cannot be changed without a restart.

The Server System Variables page states if variables are dynamic or static.

Scope

A global system variable is one that affects the general behavior of MariaDB. For example innodb_buffer_pool_size determines the size of the InnoDB buffer pool, which is used by read and write operations, no matter which user issued them. A session system variable is one that affects MariaDB behavior for the current connection; changing it will not affect other connected users, or future connections from the current user.

A variable could exist in both the global and session scopes. In this case, the session value is what affects the current connection. When a user connects, the current global value is copied to the session scope. Changing the global value afterward will not change existing connections.

The Server System Variables page states the scope of each variable.

Global variables and some session variables can only be modified by a user with the SUPER privilege (typically root).

Syntax

To see the value of a system variable:

-- global variables: SELECT @@global.variable_name; -- session variables: SELECT @@session.variable_name; -- or just use the shortcut: SELECT @@variable_name;

A longer syntax, which is mostly useful to get multiple variables, makes use of the same pattern syntax that is used by the LIKE operator:

-- global variables whose name starts with 'innodb': SHOW GLOBAL VARIABLES LIKE 'innodb%'; -- session variables whose name starts with 'innodb': SHOW SESSION VARIABLES LIKE 'innodb%'; SHOW VARIABLES LIKE 'innodb%';

To modify the global or session value of a dynamic variable:

SET @@global.variable_name = 'new 'value'; SET @@session.variable_name = 'new 'value';

Notice that if we modify a global variable in this way, the new value will be lost at server restart. For this reason we probably want to change the value in the configuration file too.

For further information see:

The SET statement.
The SHOW VARIABLES statement.

Setting System Variables with Startup Parameters

System variables can be set at server startup without writing their values into a configuration file. This is useful if we want a value to be set once, until we change it or restart MariaDB. Values passed in this way override values written in the configuration files.

The general rule is that every global variable can be passed as an argument of mysqld by prefixing its name with -- and by replacing every occurrence of _ with - in its name.

For example, to pass bind_address as a startup argument:

mysqld --bind-address=127.0.0.1

Debugging Configuration

Mistyping a variable can prevent MariaDB from starting. We cannot set a variable that doesn't exist in the MariaDB version in use. In these cases, an error is written in the error log.

Having several configuration files and configuration groups, as well as being able to pass variables as command-line arguments, brings a lot of flexibility but can sometimes be confusing. When we are unsure about which values will be used, we can run:

mysqld --print-defaults

Status Variables

MariaDB status variables and some system tables allow external tools to monitor a server, building graphs on how they change over time, and allow the user to inspect what is happening inside the server.

Status variables cannot be directly modified by the user. Their values indicate how MariaDB is operating. Their scope can be:

Global, meaning that the value is about some MariaDB activity.
Session, meaning that the value measures activities taking place in the current session.

Many status variables exist in both scopes. For example,Cpu_time at global level indicates how much time the CPU was used by the MariaDB process (including all user sessions and all the background threads). At session level, it indicates how much time the CPU was used by the current session.

The status variables created by a plugin, usually, use the plugin name as a prefix.

The SHOW STATUS statement prints the values of the status variables that match a certain pattern.

-- Show all InnoDB global status variables SHOW GLOBAL STATUS LIKE 'innodb%'; -- Show all InnoDB session status variables SHOW SESSION STATUS LIKE 'innodb%'; SHOW STATUS LIKE 'innodb%'; -- Show global variables that contain the "size" substring: SHOW GLOBAL STATUS LIKE '%size%';

Some status variables values are reset when FLUSH STATUS is executed. A possible use:

DELIMITER || BEGIN NOT ATOMIC SET @i = 0; WHILE @i < 60 DO SHOW GLOBAL STATUS LIKE 'Com_select'; FLUSH STATUS; DO SLEEP(1); SET @i = @i + 1; END WHILE; END ||

[스크랩] MariaDB에서 테이블스페이스란?

99iberty — Thu, 4 Nov 2021 23:21:25 +0900

https://hobogi.tistory.com/entry/MariaDB%EC%97%90%EC%84%9C-%ED%85%8C%EC%9D%B4%EB%B8%94%EC%8A%A4%ED%8E%98%EC%9D%B4%EC%8A%A4%EB%9E%80

Oracle DBA로 익숙해져 있어서 그런지, MariaDB에서의 Tablespace가 명확히 와닿지 않는다.

오라클은 흔히 알다시피 논리적 개념으로 1개의 테이블스페이스에 1개 이상의 데이터파일들이 매핑될 수 있다.

분명 오라클처럼 Tablespace는 존재하는 것 같은데 MariaDB의 테이블스페이스는 어떻게 다를까?

우리가 트랜잭션을 제대로 관리하기 위해 쓰는 기본 데이터베이스 엔진은 InnoDB이므로 InnoDB에서의 테이블스페이스에 대해 정리해보고자 한다.

InnoDB 스토리지 엔진을 사용하는 테이블은 "테이블스페이스"라는 데이터파일 디스크에 기록된다.

각 테이블스페이스는 하나 이상의 InnoDB, 테이블과 관련된 인덱스 데이터를 포함할 수가 있다.

InnoDB에서는 아래와 같이 크게 3가지로 테이블스페이스를 분류 할 수 있다.

- InnoDB SYSTEM 테이블스페이스

- InnoDB file-per-table 테이블스페이스

- InnoDB Temporary 테이블스페이스

InnoDB의 아키텍쳐 측면에서 보면 아래 그림과 같다.

mysql 5.7에서의 innodb architecture

* InnoDB SYSTEM 테이블스페이스 :

내용

크기 변경
InnoDB SYSTEM 테이블스페이스 내의 시스템 테이블

InnoDB가 시스템과 관련된 정보를 특정 테이블이 아닌 전체로 저장해야하는 경우, 기록하는 특정 파일이 바로 시스템 테이블스페이스이다. 기본적으로 ibdata1파일이며 데이터 디렉토리에 위치한 파일이다.

(datadir 또는 innodb_data_home_dir 시스템 변수로 정의 됨 )

InnoDB는 시스템 테이블스페이스를 사용하여 데이터 딕셔너리, 버퍼의 변경분, undo log를 저장한다.

innodb_data_file_path 시스템 변수를 설정하여 시스템 테이블스페이스 파일 이름, 크기 및 기타 옵션을 정의할 수 있다. 이 시스템 변수는 커맨드라인 인수로 mysqld에서 지정 하거나 옵션 파일 의 서버 그룹에서 지정할 수 있다 . 예를 들면 다음과 같다.

[mariadb]
...
innodb_data_file_path = ibdata1 : 50M : autoextend

이 시스템 변수의 기본값은 ibdata1 file이자, 최소 크기 12M, autoextend 속성이 활성화된 상태이다.

크기 변경

InnoDB는 기본적으로 ibdata1 시스템 테이블스페이스의 파일에 12M을 할당한다. 보통은 충분하지만 충분하지 않을수도 있다. MariaDB를 사용하다 보면 시스템 테이블 공간에 대한 할당이 너무 작거나 디스크에 비해 너무 커질 수도 있다.

다행히 나중에 크기를 조정할 수 있다.

크기 늘리기

innodb_data_file_path시스템 변수를 설정하여 각 파일의 크기를 정의 할 수 있다.

더 큰 시스템 테이블스페이스가 필요한 경우 autoextend옵션을 마지막 값에 추가하면 된다.

[mariadb]
...
innodb_data_file_path = ibdata1 : 12M; ibdata2 : 50M : autoextend

이 구성에서 시스템 테이블스페이스가 지정한 크기보다 초과하여 할당하게 되면 InnoDB는 파일 크기를 늘린다.

autoextend옵션은 innodb_autoextend_increment 시스템 변수를 설정할 수 있다.

크기 감소

InnoDB 시스템 테이블스페이스의 크기를 줄이는 과정은 크기를 늘리는 것보다 조금 더 복잡하다.

MariaDB를 사용하면 테이블스페이스 파일 자체에서 데이터를 제거할 수 없다.

대신 테이블스페이스 파일 자체를 삭제 한 다음, 백업에서 데이터베이스를 복원해야 한다.

백업 유틸리티 mysqldump는 데이터베이스를 재생성하는 데 필요한 SQL문이 포함된 백업 파일을 생성한다.

결과적으로, 최소 데이터로 데이터베이스를 복원한다.

mysqldump를 사용해서 (mysql 데이터베이스의 시스템 테이블을 포함하여 모든 InnoDB 데이터베이스 테이블) 백업합니다.

information_schema를 사용하면 사용중인 것을 찾을 수 있습니다.

SELECT TABLE_NAME FROM INFORMATION_SCHEMA.TABLES
WHERE TABLE_SCHEMA = 'mysql' AND ENGINE = 'InnoDB' ;

InnoDB 만 사용하는 경우 모든 데이터베이스와 테이블을 쉽게 백업 할 수 있습니다.

$ mysqldump -u root -p --all-databases > full-backup.sql

그런 다음 MariaDB 서버를 중지하고 InnoDB 테이블 스페이스 파일을 제거합니다.

데이터 디렉토리 또는 InnoDB의 데이터를 홈 디렉토리에서 모든 ibdata와 ib_log 파일들을 삭제하십시요.

(확장자가 .ibd, .frm과 같은 모든 파일들)

이 작업이 완료되면 서버를 다시 시작하고 덤프 파일을 가져옵니다.

$ mysql -u root -p < full-backup.sql

InnoDB 시스템 테이블스페이스 내의 System table들

InnoDB는 InnoDB 시스템 테이블스페이스 내에 시스템 테이블을 생성합니다 :

SYS_DATAFILES
SYS_FOREIGN
SYS_FOREIGN_COLS
SYS_TABLESPACES
SYS_VIRTUAL
SYS_ZIP_DICT
SYS_ZIP_DICT_COLS

위 테이블은 쿼리할 수 없습니다. 하지만 information_schema 데이터베이스의 INNODB_SYS_TABLES과 같은 테이블에서 확인할 수 있습니다.

* InnoDB file-per-table 테이블스페이스 :

내용

file-per-table의 위치
transportable tablespace copy
테이블 스페이스 암호화
그 외...

InnoDB 스토리지 엔진을 사용하여 테이블을 생성하면 해당 테이블에 기록된 데이터는 파일시스템의 tablespace라는 데이터 파일에 저장됩니다. tablespace file에는 데이터와 인덱스가 모두 포함됩니다.

innodb_file_per_table=ON 설정을 하면, InnoDB는 InnoDB의 테이블 당 한개의 tablespace file을 사용합니다.

이 때 tablespace file의 확장자는 .ibd입니다.

innodb_file_per_table=OFF 설정을 하면, InnoDB는 InnoDB의 시스템 테이블스페이스를 모든 테이블에 저장합니다.

MySQL 5.7 이상의 InnoDB 버전은 CREATE TABLESPACE 명령어와 general tablespace라는 유형의 테이블스페이스를 지원합니다.

하지만 MariaDB 서버의 InnoDB 버전은 CREATE TABLESPACE 명령어와 general tablespace 유형의 테이블스페이스를 지원하지 않습니다.

File-Per-Table 테이블스페이스 위치

기본적으로 InnoDB의 File-Per-Table 테이블스페이스는 datadir 시스템 변수에 의해 정의되는 시스템의 데이터 디렉토리에 위치하게 됩니다. InnoDB 데이터를 다른 스토리지 엔진의 데이터와 별도로 저장하려면 innodb_data_home_dir시스템 변수를 설정하여 이를 변경하십시오.

전용 경로에 저장해야 하는 특정 테이블 스페이스가 있는 경우 테이블 생성시에 DATA DIRECTORY 옵션을 사용하여 위치를 설정할 수 있습니다.

예를 들어

CREATE TABLE test.t1
( id INT PRIMARY KEY AUTO_INCREMENT,
name VARCHAR(50)
) ENGINE=InnoDB
DATA DIRECTORY = "/data/contact";

데이터 디렉토리를 작성하면 MariaDB는 해당 디렉토리 내에 File-Per-Table 테이블스페이스가 생성합니다.

유닉스 계열 운영체제에서 ls 명령을 사용하여 파일을 볼 수 있습니다.

# ls -al /data/contact/test
drwxrwx--- 2 mysql mysql 4096 Dec 8 18:46 .
drwxr-xr-x 3 mysql mysql 4096 Dec 8 18:46 ..
-rw-rw---- 1 mysql mysql 98304 Dec 8 20:41 t1.ibd

MariaDB 서버 프로세스 (보통 mysql)를 실행하는 시스템 사용자는 주어진 경로에 대해 쓰기 권한이 있어야 합니다.

[스크랩] SQL lock / blocking / dead lock 간단정리

99iberty — Thu, 22 Jul 2021 15:58:34 +0900

https://chrisjune-13837.medium.com/db-lock-%EB%9D%BD%EC%9D%B4%EB%9E%80-%EB%AC%B4%EC%97%87%EC%9D%B8%EA%B0%80-d908296d0279

[DB] Lock이란?

DBMS에서 데이터의 일관성을 보장하기 위한 기본적인 방법인 Lock에 대해서 알아봅니다

chrisjune-13837.medium.com

DBMS에서 데이터의 일관성을 보장하기 위한 기본적인 방법인 Lock에 대해서 알아봅니다.

Lock이란?

데이터의 일관성을 보장하기 위한 방법입니다. 오라클과 같이 고가의 DBMS를 사용하는 이유가 데이터의 무결성과 일관성을 유지하는 능력이 뛰어나기 때문입니다.

Lock의 종류는?

Lock은 상황에 따라서 크게 두가지로 나누어 집니다.

Shared Lock(공유 Lock 또는 Read Lock)

보통 데이터를 읽을 때 사용합니다. 원하는 데이터에 lock을 걸었지만 다른 세션에서 읽을 수 있습니다. 공유Lock을 설정한 경우 추가로 공유Lock을 설정할 수 있지만, 배타적 Lock은 설정할 수 없습니다. 즉, 내가 보고 있는 데이터는 다른 사용자가 볼 수 있지만, 변경할 수는 없습니다.

2. Exclusive Lock(배타적 Lock 또는 Write lock)

보통 데이터를 변경할 때 사용합니다. 이름에서 느껴지는 것 처럼 해당 Lock이 해제되기 전까지는, 다른 공유Lock, 배타적Lock을 설정할 수 없습니다. 즉, 읽기 기와 쓰기가 불가능하다는 의미입니다.

Blocking ?

경합으로 인한 상태인 블로킹에 대하여 알아봅니다.

Blocking

블로킹은 Lock들의 경합(Race condition이라고도 합니다)이 발생하여 특정 세션이 작업을 진행하지 못하고 멈춰 선 상태를 의미합니다. 공유Lock과 배타적Lock 또는 배타적Lock과 배타적Lock끼리 블로킹이 발생할 수 있습니다. 이를 해결하는 방법은 Transaction commit 또는 rollback 뿐입니다.

경합이 발생할 때, 먼저 Lock을 설정한 트랜젝션을 기다려야하기 때문에, 이런 현상이 반복되면 빠른 서비스를 제공할 수 없습니다.

해결방안

SQL 문장에 가장 빠르게 실행되도록 리펙토링하는 것이 가장 기본이며 효과적인 방법입니다.
트랜젝션을 가능한 짧게 정의하면 경합을 줄일 수 있습니다.
동일한 데이터를 동시에 변경하는 작업을 하지 않도록 설계하는 것이 좋습니다. 또한 트랜젝션이 활발한 주간에는 대용량 갱신 작업을 수행하면 안됩니다.
대용량작업이 불가피할 경우, 작업단위를 쪼개거나 lock_timeout을 설정하여 해당 Lock의 최대시간을 설정할 수 있습니다.

set lock_timeout 3000

Dead Lock이란?

deadlock은 트랜젝션간의 교착상태를 의미합니다. 두개의 트랜젝션간에 각각의 트랜젝션이 가지고 있는 리소스의 Lock을 획득하려고 할 때 발생합니다.

Dead Lock Example1

가장 흔히 떠올릴 수 있는 deadlock 상황입니다. 1번 트랜젝션에서 2번 리소스의 잠금을 획득, 2번 트랜젝션에서는 1번 리소스의 잠금을 획득한 상태입니다. 이때, 동시에 상대방의 데이터를 엑세스하려고 할때 기존의 Lock이 해제될 때까지 기다리게 되는 상황입니다.

Dead Lock Example2

두번째 예는, 1번 트랜젝션이 공유Lock을 설정하고 Sleep에 빠졌습니다. 이때 2번 트랜젝션은 배타적Lock을 설정하려고 할때, 무기한 기다리게되는 교착상태에 빠지게 됩니다.

Postgresql에서는 자동적으로 deadlock을 인지하고 교착상태를 해결하기 위하여 하나의 트랜젝션을 취소시킵니다. 그러나, 어떤 트랜젝션이 취소될 지 판단하기는 어렵습니다.

Lock Level & Escalation이란?

SQL 명령어에 따라서 Lock의 설정대상이 데이터 row일지 database일지 나누어집니다.

Row level, 변경하려는 row에만 lock을 설정하는 것을 의미합니다.
Page level, 변경하려는 row가 담긴 데이터 page (또는 인덱스 페이지)에 lock을 설정합니다. 같은 페이지에 속한 row들은 변경작업과 무관하게 모두 lock에 의해 잠깁니다.
Tabel level, 테이블과 인덱스에 모두 잠금을 설정합니다. Select table, Alter table, Vacuum, Refresh, Index, Drop, Truncate 등의 작업에서 해당레벨의 락이 설정됩니다.
Database level, 데이터베이스를 복구하거나 스키마를 변경할 때 발생합니다.

Escalation

Lock리소스가 임계치를 넘으면 Lock 레벨이 확장되는 것을 의미합니다. Lock 레벨이 낮을 수록 동시성이 좋아지지만, 관리해야할 Lock이 많아지기 때문에 메모리 효율성은 떨어집니다. 반대로 Lock레벨이 높을 수록 관리리소스는 낮지만, 동시성은 떨어집니다.

참고링크

[스크랩] Can DBCC SHRINKFILE Cause Blocking in SQL Server?

99iberty — Thu, 22 Jul 2021 15:57:17 +0900

https://www.brentozar.com/archive/2015/04/dbcc-shrinkfile-blocking-locks-in-sql-server/

Can DBCC SHRINKFILE Cause Blocking in SQL Server? - Brent Ozar Unlimited®

It sure can. The lock risks of shrinking data files in SQL Server aren’t very well documented. Many people have written about shrinking files being a bad regular practice— and that’s totally true. But sometimes you may need to run a one-time operatio

www.brentozar.com

Last Updated January 11, 2018

Kendra Little

Locking, Blocking, and Isolation Levels, SQL Server

36Comments

It sure can.

The lock risks of shrinking data files in SQL Server aren’t very well documented. Many people have written about shrinking files being a bad regular practice— and that’s totally true. But sometimes you may need to run a one-time operation if you’ve been able to clear out or archive a lot of data. And you might wonder what kind of pains shrinking could cause you.

One pain it could cause you is blocking and lock waits. It’s easy to reproduce, here’s how.

A Recipe for LCK_M_X Lock Waits and Blocking

I was able to easily reproduce blocking in my restored copy of the StackOverflow database on SQL Server 2014 by doing a few things.

I rebuilt a non-clustered index on the Posts table with this command:

1 2	ALTER INDEX [kl_Posts_OwnerUserId_CreationDate] on Posts REBUILD WITH (ONLINE=ON); GO

This ate up some space, and gave shrink something to do!

I then started a data modification and left an open transaction running:

1 2	BEGIN TRAN DELETE FROM dbo.Posts where Id=16;

And then, in another session, I started to shrink the data file:

1 2	DBCC SHRINKFILE ('StackOverflow'); GO

Behold, Blocking!

Shrink starts up and slogs its way through things, and soon enough, lock waits appear. This view is from Adam Machanic‘s sp_WhoIsActive:

I’ll take LCK_M_X Waits for 15,000, Alex!

If I run that with @get_locks=1, I can see details on exactly what lock SHRINKFILE is after:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19

Yep, it’s waiting on a page lock in kl_Posts_OwnerUserId_CreationDate.

But How Many Locks did SHRINKFILE Really Need?

Maybe it needed just a few locks… right?

Well, I ran a trace against my session that did that shrinkfile command, and here’s the number of locks by database and type that it caught…

Lock mode: XTreme

And this was a FAST run, except for that blocking!

Shrinking files is painful

It can actually be worse than this — on a few runs, I was able to reproduce blocking with SCH_M locks that could cause even nastier blocking chains.

Shrinking is slow, tedious work. It eats CPU, steals your IO, and always takes longer than you want. And if it has work to do and you’ve got users or processors modifying that data, lock contention ensues.

Worst of all, at the end of shrinking a database, nobody will ever thank you. Be careful out there, and if you must run it make sure you don’t leave it unattended.

Brent says: whoa, I never even thought about that. Maybe we should design indexes to improve DBCC SHRINKDB speed.

Erik says: if you do this more than once, I will come find you.

System engineer

[스크랩] SQL SERVER 스키마란?

Understanding Logging and Recovery in SQL Server

Contents

[스크랩] Lock 확인, 조치방법

[스크랩] 오픈 트랜잭션 쿼리가 있는 DB 찾아보기

[스크랩] Lazy Log Truncation - Clearing of SQL transaction log VLF status deferred

Linux VM root로 로그인하기

[스크랩] Understanding MariaDB Architecture

Understanding MariaDB Architecture

Contents

Storage Engines

InnoDB

Primary Key and Indexes

Tablespaces

Transaction Logs

InnoDB Buffer Pool

InnoDB Background Threads

Checksums and Doublewrite Buffer

Aria

Databases

System Databases

Default Database

The Binary Log

Plugins

Thread Pool

Configuration

Configuration Files

Dynamic and Static Variables

Scope

Syntax

Setting System Variables with Startup Parameters

Debugging Configuration

Status Variables

[스크랩] MariaDB에서 테이블스페이스란?

* InnoDB SYSTEM 테이블스페이스 :

내용

크기 변경

크기 늘리기

크기 감소

InnoDB 시스템 테이블스페이스 내의 System table들

* InnoDB file-per-table 테이블스페이스 :

내용

File-Per-Table 테이블스페이스 위치

[스크랩] SQL lock / blocking / dead lock 간단정리

Lock이란?

Lock의 종류는?

Blocking ?

Dead Lock이란?

Lock Level & Escalation이란?

[스크랩] Can DBCC SHRINKFILE Cause Blocking in SQL Server?

A Recipe for LCK_M_X Lock Waits and Blocking

Behold, Blocking!

But How Many Locks did SHRINKFILE Really Need?

Shrinking files is painful